20 KiB

Raw Blame History

/user_testing Command

Main UI/browser testing command for executing Epic testing workflows using Claude-native subagent orchestration with structured BMAD reporting. This command is for UI testing ONLY.

Command Usage

/user_testing [epic_target] [options]

Parameters

epic_target - Target for testing (epic-3.3, story-3.2, custom document path)
--mode [automated|interactive|hybrid] - Testing execution mode (default: hybrid)
--cleanup [session_id] - Clean up specific session
--cleanup-older-than [days] - Remove sessions older than specified days
--archive [session_id] - Archive session to permanent storage
--list-sessions - List all active sessions with status
--include-size - Include session sizes in listing
--resume [session_id] - Resume interrupted session from last checkpoint

Examples

# Clean up old sessions
/user_testing --cleanup-older-than 7

# List all active sessions with sizes
/user_testing --list-sessions --include-size

# Resume interrupted session
/user_testing --resume epic-3.3_hybrid_20250829_143000_abc123

CRITICAL: UI/Browser Testing Only

This command executes UI/browser testing EXCLUSIVELY. When invoked:

ALWAYS use chrome-browser-executor for Phase 3 test execution
Focus on browser-based user interface testing

Command Implementation

You are the main testing orchestrator for the BMAD testing framework. You coordinate the execution of all testing agents using Task tool orchestration with markdown-based communication for seamless agent coordination and improved accessibility.

Execution Workflow

Phase 0: UI Discovery & User Clarification (NEW)

User Interface Analysis:

Spawn ui-test-discovery agent to analyze project UI
- Discovers user interfaces and entry points
- Identifies user workflows and interaction patterns
- Generates UI_TEST_DISCOVERY.md with clarifying questions
Present UI options to user for clarification
- Display discovered user interfaces and workflows
- Ask specific questions about testing objectives
- Get user confirmation of testing scope and personas
Finalize UI test objectives based on user responses
- Create UI_TEST_OBJECTIVES.md with confirmed testing plan
- Define specific user workflows to validate
- Set clear success criteria from user perspective

Phase 1: Session Initialization

Markdown-Based Setup:

Generate unique session ID: {target}_{mode}_{date}_{time}_{hash}
Create session directory structure optimized for markdown files
Copy UI test objectives to session directory
Validate UI access and testing prerequisites

Directory Structure:

workspace/testing/sessions/{session_id}/
├── UI_TEST_DISCOVERY.md        # Generated by ui-test-discovery
├── UI_TEST_OBJECTIVES.md       # Based on user clarification responses
├── REQUIREMENTS.md             # Generated by requirements-analyzer (from UI objectives)
├── SCENARIOS.md                # Generated by scenario-designer (UI-focused)
├── BROWSER_INSTRUCTIONS.md     # Generated by scenario-designer (UI automation)
├── EXECUTION_LOG.md            # Generated by playwright-browser-executor
├── EVIDENCE_SUMMARY.md         # Generated by evidence-collector
├── BMAD_REPORT.md              # Generated by bmad-reporter (UI testing results)
└── evidence/                   # PNG screenshots and UI interaction data
    ├── ui_workflow_001_step_1.png
    ├── ui_workflow_001_step_2.png
    ├── ui_workflow_002_complete.png
    └── user_interaction_metrics.json

Phase 2: UI Requirements Processing

UI-Focused Requirements Chain:

Spawn requirements-analyzer agent via Task tool
- Input: UI_TEST_OBJECTIVES.md (user-confirmed UI testing goals)
- Output: REQUIREMENTS.md with UI-focused requirements analysis
Spawn scenario-designer agent via Task tool
- Input: REQUIREMENTS.md + UI_TEST_OBJECTIVES.md
- Output: SCENARIOS.md (UI workflows) + BROWSER_INSTRUCTIONS.md (UI automation)
Wait for markdown files and validate UI test scenarios are ready

Phase 3: UI Test Execution

UI-Focused Browser Testing:

Spawn chrome-browser-executor agent via Task tool # Use chrome-browser-executor for UI testing
- Input: BROWSER_INSTRUCTIONS.md (UI automation steps)
- Focus: User interface interactions, workflows, and experience validation
- Output: EXECUTION_LOG.md with comprehensive UI testing results
Spawn interactive-guide agent (if hybrid/interactive mode)
- Input: SCENARIOS.md (UI workflows for manual testing)
- Focus: User experience validation and usability assessment
- Output: Manual UI testing results appended to execution log
Monitor UI testing progress through evidence file creation

Phase 4: UI Evidence Collection & Reporting

UI Testing Results Processing:

Spawn evidence-collector agent via Task tool
- Input: EXECUTION_LOG.md + UI evidence files (screenshots, interactions)
- Focus: UI testing evidence organization and accessibility validation
- Output: EVIDENCE_SUMMARY.md with UI testing evidence analysis
Spawn bmad-reporter agent via Task tool
- Input: EVIDENCE_SUMMARY.md + UI_TEST_OBJECTIVES.md + REQUIREMENTS.md
- Focus: UI testing business impact and user experience assessment
- Output: BMAD_REPORT.md (executive UI testing deliverable)

UI-Focused Task Tool Orchestration

Phase 0: UI Discovery & User Clarification

task_ui_discovery = Task(
    subagent_type="ui-test-discovery",
    description="Discover UI and clarify testing objectives",
    prompt=f"""
    Analyze this project's user interface and generate testing clarification questions.
    
    Project Directory: {project_dir}
    Session Directory: {session_dir}
    
    Perform comprehensive UI discovery:
    1. Read project documentation (README.md, CLAUDE.md) for UI entry points
    2. Glob source directories to identify UI frameworks and patterns
    3. Grep for URLs, user workflows, and interface descriptions
    4. Discover how users access and interact with the system
    5. Generate UI_TEST_DISCOVERY.md with:
       - Discovered UI entry points and access methods
       - Available user workflows and interaction patterns
       - Context-aware clarifying questions for user
       - Recommended UI testing approaches
    
    FOCUS EXCLUSIVELY ON USER INTERFACE - no APIs, databases, or backend analysis.
    Output: UI_TEST_DISCOVERY.md ready for user clarification
    """
)

# Present discovery results to user for clarification
print("🖥️ UI Discovery Complete! Please review and clarify your testing objectives:")
print("=" * 60)
display_ui_discovery_results()
print("=" * 60)

# Get user responses to clarification questions
user_responses = collect_user_clarification_responses()

# Generate final UI test objectives based on user input
task_ui_objectives = Task(
    subagent_type="ui-test-discovery",
    description="Finalize UI test objectives",
    prompt=f"""
    Create final UI testing objectives based on user responses.
    
    Session Directory: {session_dir}
    UI Discovery: {session_dir}/UI_TEST_DISCOVERY.md
    User Responses: {user_responses}
    
    Generate UI_TEST_OBJECTIVES.md with:
    1. Confirmed UI testing scope and user workflows
    2. Specific user personas and contexts for testing
    3. Clear success criteria from user experience perspective
    4. Testing environment and access requirements
    5. Evidence and documentation requirements
    
    Transform user clarifications into actionable UI testing plan.
    Output: UI_TEST_OBJECTIVES.md ready for requirements analysis
    """
)

Phase 2: UI Requirements Analysis

task_requirements = Task(
    subagent_type="requirements-analyzer",
    description="Extract UI testing requirements from objectives",
    prompt=f"""
    Transform UI testing objectives into structured testing requirements using markdown communication.
    
    Session Directory: {session_dir}
    UI Test Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
    
    Process user-confirmed UI testing objectives:
    1. Read UI_TEST_OBJECTIVES.md for user-confirmed testing goals
    2. Extract UI-focused acceptance criteria and user workflow requirements
    3. Transform user personas and success criteria into testable requirements
    4. Identify UI testing dependencies and environment needs
    5. Write UI-focused REQUIREMENTS.md to session directory
    6. Ensure all requirements focus on user interface and user experience
    
    FOCUS ON USER INTERFACE REQUIREMENTS ONLY - no backend, API, or database requirements.
    Output: Complete REQUIREMENTS.md ready for UI scenario generation.
    """
)

task_scenarios = Task(
    subagent_type="scenario-designer", 
    description="Generate UI test scenarios from requirements",
    prompt=f"""
    Create UI-focused test scenarios using markdown communication.
    
    Session Directory: {session_dir}
    Requirements File: {session_dir}/REQUIREMENTS.md
    UI Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
    Testing Mode: {testing_mode}
    
    Generate comprehensive UI test scenarios:
    1. Read REQUIREMENTS.md for UI testing requirements analysis
    2. Read UI_TEST_OBJECTIVES.md for user-confirmed workflows and personas
    3. Design UI test scenarios covering all user workflows and acceptance criteria
    4. Create detailed SCENARIOS.md with step-by-step user interaction procedures
    5. Generate BROWSER_INSTRUCTIONS.md with Playwright MCP commands for UI automation
    6. Include UI coverage analysis and user workflow traceability
    
    FOCUS EXCLUSIVELY ON USER INTERFACE TESTING - no API, database, or backend scenarios.
    Output: SCENARIOS.md and BROWSER_INSTRUCTIONS.md ready for UI test execution.
    """
)

Phase 3: UI Test Execution

task_ui_browser_execution = Task(
    subagent_type="chrome-browser-executor",  # MANDATORY: Always use chrome-browser-executor for UI testing
    description="Execute automated UI testing with Chrome DevTools",
    prompt=f"""
    Execute comprehensive UI testing using Chrome DevTools MCP with markdown communication.
    
    Session Directory: {session_dir}
    Browser Instructions: {session_dir}/BROWSER_INSTRUCTIONS.md
    UI Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
    Evidence Directory: {session_dir}/evidence/
    
    Execute all UI test scenarios with user experience focus:
    1. Read BROWSER_INSTRUCTIONS.md for detailed UI automation procedures
    2. Execute all user workflows using Chrome DevTools MCP tools
    3. Capture PNG screenshots of each user interaction step
    4. Monitor user interface responsiveness and performance
    5. Document user experience issues and accessibility problems
    6. Generate comprehensive EXECUTION_LOG.md focused on UI validation
    7. Save all evidence in accessible formats for UI analysis
    
    FOCUS ON USER INTERFACE TESTING - validate UI behavior, user workflows, and experience.
    Output: Complete EXECUTION_LOG.md with UI testing evidence ready for collection.
    """
)

Phase 4: UI Evidence & Reporting

task_ui_evidence_collection = Task(
    subagent_type="evidence-collector",
    description="Collect and organize UI testing evidence",
    prompt=f"""
    Aggregate UI testing evidence into comprehensive summary using markdown communication.
    
    Session Directory: {session_dir}
    Execution Results: {session_dir}/EXECUTION_LOG.md
    Evidence Directory: {session_dir}/evidence/
    UI Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
    
    Collect and organize UI testing evidence:
    1. Read EXECUTION_LOG.md for comprehensive UI test results
    2. Catalog all UI evidence files (screenshots, user interaction logs, performance data)
    3. Verify evidence accessibility (PNG screenshots, readable formats)
    4. Create traceability matrix mapping user workflows to evidence
    5. Generate comprehensive EVIDENCE_SUMMARY.md focused on UI validation
    
    FOCUS ON UI TESTING EVIDENCE - user workflows, interface validation, experience assessment.
    Output: Complete EVIDENCE_SUMMARY.md ready for UI testing report.
    """
)

task_ui_bmad_reporting = Task(
    subagent_type="bmad-reporter",
    description="Generate UI testing executive report",
    prompt=f"""
    Create comprehensive UI testing BMAD report using markdown communication.
    
    Session Directory: {session_dir}
    Evidence Summary: {session_dir}/EVIDENCE_SUMMARY.md
    UI Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
    Requirements Context: {session_dir}/REQUIREMENTS.md
    
    Generate executive UI testing analysis:
    1. Read EVIDENCE_SUMMARY.md for comprehensive UI testing evidence
    2. Read UI_TEST_OBJECTIVES.md for user-confirmed success criteria
    3. Read REQUIREMENTS.md for UI requirements context
    4. Synthesize UI testing findings into business impact assessment
    5. Develop user experience recommendations with implementation timelines
    6. Generate executive BMAD_REPORT.md focused on UI validation results
    
    FOCUS ON USER INTERFACE TESTING OUTCOMES - user experience, UI quality, workflow validation.
    Output: Complete BMAD_REPORT.md ready for executive review of UI testing results.
    """
)

Markdown Communication Advantages

Enhanced Agent Coordination:

Human Readable: All coordination files in markdown format for easy inspection
Standard Templates: Consistent structure across all testing sessions
Accessibility: Evidence and reports accessible in any text editor or browser
Version Control: All session files can be tracked with git
Debugging: Clear audit trail through markdown file progression

Technical Benefits:

Simplified Communication: No complex YAML/JSON parsing required
Universal Accessibility: PNG screenshots viewable in any image software
Better Error Recovery: Markdown files can be manually edited if needed
Improved Collaboration: Human reviewers can validate agent outputs
Documentation: Session becomes self-documenting with markdown files

Key Framework Improvements

Chrome DevTools MCP Integration:

Robust Browser Automation: Direct Chrome DevTools integration for reliable UI testing
Enhanced Screenshot Capture: High-quality PNG screenshots with element-specific capture
Performance Monitoring: Comprehensive network and timing analysis via DevTools
Error Handling: Better failure recovery with detailed error capture
Page Management: Advanced page and tab management capabilities

Evidence Management:

Accessible Formats: All evidence in standard, universally accessible formats
Organized Storage: Clear directory structure with descriptive file names
Quality Assurance: Evidence validation and integrity checking
Comprehensive Coverage: Complete traceability from requirements to evidence

Session Management Features

Session Lifecycle Management

Session States:
  - initialized: Session created, configuration set
  - phase_0: Target document loaded and analyzed
  - phase_1: Requirements extraction in progress  
  - phase_2: Test execution in progress
  - phase_3: Evidence collection and reporting in progress
  - completed: All phases successful, results available
  - failed: Unrecoverable error, session terminated
  - archived: Session completed and moved to archive

Cleanup and Maintenance

Automatic Cleanup:
  - Time-based: Remove sessions > 72 hours old
  - Size-based: Archive sessions > 100MB 
  - Status-based: Remove failed sessions > 24 hours old
  - Evidence preservation: Compress successful sessions > 30 days

Manual Cleanup Commands:
  - /user_testing --cleanup {session_id}
  - /user_testing --cleanup-older-than 7
  - /user_testing --archive {session_id}
  - /user_testing --list-sessions --include-size

Error Recovery and Resume

Resume Capabilities:
  - Checkpoint detection: Identify last successful phase
  - State reconstruction: Rebuild session context from files
  - Partial retry: Continue from interruption point
  - Agent restart: Re-spawn failed agents with existing context
  
Recovery Procedures:
  - Phase 1 failure: Retry requirements extraction
  - Phase 2 failure: Switch to manual-only mode if browser automation fails
  - Phase 3 failure: Regenerate reports from existing evidence
  - Session corruption: Rollback to last successful checkpoint

Integration with Existing Infrastructure

Story 3.2 Dependency Integration

Prerequisites:
  - requirements-analyzer agent: Available and tested
  - scenario-designer agent: Available and tested  
  - validation-planner agent: Available and tested
  - Session coordination patterns: Proven in Story 3.2 tests
  
Integration Pattern:
  1. Use existing Story 3.2 agents for phase 1 processing
  2. Extend session coordination to phases 2-3
  3. Maintain file-based communication compatibility
  4. Preserve session schema and validation patterns

Quality Gates and Validation

Quality Gates:
  Phase 1 Gates:
    - Requirements extraction accuracy ≥ 95%
    - Test scenario generation completeness ≥ 90%
    - Validation checkpoint coverage = 100%
    
  Phase 2 Gates:  
    - Test execution completion ≥ 70% scenarios
    - Evidence collection success ≥ 90%
    - Performance within 5-minute limit
    
  Phase 3 Gates:
    - Evidence package validation = 100%
    - BMAD report generation = Complete
    - Coverage analysis accuracy ≥ 95%

Performance and Monitoring

Performance Targets

Phase 1: ≤ 2 minutes for requirements processing
Phase 2: ≤ 5 minutes for test execution
Phase 3: ≤ 1 minute for reporting
Total Session: ≤ 8 minutes for complete epic testing

Monitoring and Logging

Real-time session status updates
Agent execution progress tracking
Error detection and alerting
Performance metrics collection
Resource usage monitoring

Command Output

Success Output

✅ BMAD Testing Session Completed Successfully

Session ID: epic-3.3_hybrid_20250829_143000_abc123
Target: Epic 3.3 - Test Execution & BMAD Reporting Engine
Mode: Hybrid (Automated + Manual)
Duration: 4.2 minutes

📊 Results Summary:
- Acceptance Criteria Coverage: 85.7% (6/7 ACs)
- Test Scenarios Executed: 12/15
- Evidence Files Generated: 41
- Issues Found: 2 Major, 3 Minor
- Recommendations: 8 actionable items

📋 Reports Generated:
- BMAD Brief: workspace/testing/sessions/{session_id}/phase_3/bmad_brief.md
- Recommendations: workspace/testing/sessions/{session_id}/phase_3/recommendations.json
- Evidence Package: workspace/testing/sessions/{session_id}/phase_2/evidence/package.json

🎯 Next Steps:
1. Review BMAD brief for critical findings
2. Implement high-priority recommendations  
3. Address browser automation reliability issues

Session archived to: workspace/testing/archive/2025-08-29/

Error Output

❌ BMAD Testing Session Failed

Session ID: epic-3.3_hybrid_20250829_143000_abc123
Target: Epic 3.3 - Test Execution & BMAD Reporting Engine
Duration: 2.1 minutes (failed in Phase 2)

🔍 Failure Analysis:
- Phase 1: ✅ Completed successfully
- Phase 2: ❌ Browser automation timeout, manual testing incomplete
- Phase 3: ⏸️ Not reached

🛠️ Recovery Options:
1. Retry with interactive-only mode: /user_testing epic-3.3 --mode interactive
2. Resume from Phase 2: /user_testing --resume epic-3.3_hybrid_20250829_143000_abc123
3. Review detailed logs: workspace/testing/sessions/{session_id}/phase_2/execution_log.json

### Browser Session Troubleshooting
If tests fail with "Browser is already in use" error:
1. **Close Chrome windows**: Look for Chrome DevTools-opened Chrome windows and close them
2. **Check page status**: Use Chrome DevTools list_pages to see active sessions
3. **Retry test**: Browser session will be available for next test

Session preserved for debugging. Use --cleanup to remove when resolved.

This command orchestrates the complete BMAD testing workflow through Claude-native Task tool coordination, providing comprehensive epic testing with structured reporting in under 8 minutes.

20 KiB Raw Blame History