ldy26098
/
BMAD-METHOD
mirror of https://github.com/bmad-code-org/BMAD-METHOD.git
1
0
Fork 0
Code Packages Projects Releases Wiki Activity
BMAD-METHOD/samples/sample-custom-modules/cc-agents-commands/commands/user-testing.md

503 lines
20 KiB
Markdown
Raw Blame History

# /user_testing Command
Main UI/browser testing command for executing Epic testing workflows using Claude-native subagent orchestration with structured BMAD reporting. This command is for UI testing ONLY.
## Command Usage
```bash
/user_testing [epic_target] [options]
```
### Parameters
- `epic_target` - Target for testing (epic-3.3, story-3.2, custom document path)
- `--mode [automated|interactive|hybrid]` - Testing execution mode (default: hybrid)
- `--cleanup [session_id]` - Clean up specific session
- `--cleanup-older-than [days]` - Remove sessions older than specified days
- `--archive [session_id]` - Archive session to permanent storage
- `--list-sessions` - List all active sessions with status
- `--include-size` - Include session sizes in listing
- `--resume [session_id]` - Resume interrupted session from last checkpoint
### Examples
```bash
# Clean up old sessions
/user_testing --cleanup-older-than 7
# List all active sessions with sizes
/user_testing --list-sessions --include-size
# Resume interrupted session
/user_testing --resume epic-3.3_hybrid_20250829_143000_abc123
```
## CRITICAL: UI/Browser Testing Only
This command executes UI/browser testing EXCLUSIVELY. When invoked:
- ALWAYS use chrome-browser-executor for Phase 3 test execution
- Focus on browser-based user interface testing
## Command Implementation
You are the main testing orchestrator for the BMAD testing framework. You coordinate the execution of all testing agents using Task tool orchestration with **markdown-based communication** for seamless agent coordination and improved accessibility.
### Execution Workflow
#### Phase 0: UI Discovery & User Clarification (NEW)
**User Interface Analysis:**
1. **Spawn ui-test-discovery** agent to analyze project UI
- Discovers user interfaces and entry points
- Identifies user workflows and interaction patterns
- Generates `UI_TEST_DISCOVERY.md` with clarifying questions
2. **Present UI options to user** for clarification
- Display discovered user interfaces and workflows
- Ask specific questions about testing objectives
- Get user confirmation of testing scope and personas
3. **Finalize UI test objectives** based on user responses
- Create `UI_TEST_OBJECTIVES.md` with confirmed testing plan
- Define specific user workflows to validate
- Set clear success criteria from user perspective
#### Phase 1: Session Initialization
**Markdown-Based Setup:**
1. Generate unique session ID: `{target}_{mode}_{date}_{time}_{hash}`
2. Create session directory structure optimized for markdown files
3. Copy UI test objectives to session directory
4. Validate UI access and testing prerequisites
**Directory Structure:**
```
workspace/testing/sessions/{session_id}/
├── UI_TEST_DISCOVERY.md # Generated by ui-test-discovery
├── UI_TEST_OBJECTIVES.md # Based on user clarification responses
├── REQUIREMENTS.md # Generated by requirements-analyzer (from UI objectives)
├── SCENARIOS.md # Generated by scenario-designer (UI-focused)
├── BROWSER_INSTRUCTIONS.md # Generated by scenario-designer (UI automation)
├── EXECUTION_LOG.md # Generated by playwright-browser-executor
├── EVIDENCE_SUMMARY.md # Generated by evidence-collector
├── BMAD_REPORT.md # Generated by bmad-reporter (UI testing results)
└── evidence/ # PNG screenshots and UI interaction data
├── ui_workflow_001_step_1.png
├── ui_workflow_001_step_2.png
├── ui_workflow_002_complete.png
└── user_interaction_metrics.json
```
#### Phase 2: UI Requirements Processing
**UI-Focused Requirements Chain:**
1. **Spawn requirements-analyzer** agent via Task tool
- Input: `UI_TEST_OBJECTIVES.md` (user-confirmed UI testing goals)
- Output: `REQUIREMENTS.md` with UI-focused requirements analysis
2. **Spawn scenario-designer** agent via Task tool
- Input: `REQUIREMENTS.md` + `UI_TEST_OBJECTIVES.md`
- Output: `SCENARIOS.md` (UI workflows) + `BROWSER_INSTRUCTIONS.md` (UI automation)
3. **Wait for markdown files** and validate UI test scenarios are ready
#### Phase 3: UI Test Execution
**UI-Focused Browser Testing:**
1. **Spawn chrome-browser-executor** agent via Task tool # Use chrome-browser-executor for UI testing
- Input: `BROWSER_INSTRUCTIONS.md` (UI automation steps)
- Focus: User interface interactions, workflows, and experience validation
- Output: `EXECUTION_LOG.md` with comprehensive UI testing results
2. **Spawn interactive-guide** agent (if hybrid/interactive mode)
- Input: `SCENARIOS.md` (UI workflows for manual testing)
- Focus: User experience validation and usability assessment
- Output: Manual UI testing results appended to execution log
3. **Monitor UI testing progress** through evidence file creation
#### Phase 4: UI Evidence Collection & Reporting
**UI Testing Results Processing:**
1. **Spawn evidence-collector** agent via Task tool
- Input: `EXECUTION_LOG.md` + UI evidence files (screenshots, interactions)
- Focus: UI testing evidence organization and accessibility validation
- Output: `EVIDENCE_SUMMARY.md` with UI testing evidence analysis
2. **Spawn bmad-reporter** agent via Task tool
- Input: `EVIDENCE_SUMMARY.md` + `UI_TEST_OBJECTIVES.md` + `REQUIREMENTS.md`
- Focus: UI testing business impact and user experience assessment
- Output: `BMAD_REPORT.md` (executive UI testing deliverable)
### UI-Focused Task Tool Orchestration
**Phase 0: UI Discovery & User Clarification**
```python
task_ui_discovery = Task(
subagent_type="ui-test-discovery",
description="Discover UI and clarify testing objectives",
prompt=f"""
Analyze this project's user interface and generate testing clarification questions.
Project Directory: {project_dir}
Session Directory: {session_dir}
Perform comprehensive UI discovery:
1. Read project documentation (README.md, CLAUDE.md) for UI entry points
2. Glob source directories to identify UI frameworks and patterns
3. Grep for URLs, user workflows, and interface descriptions
4. Discover how users access and interact with the system
5. Generate UI_TEST_DISCOVERY.md with:
- Discovered UI entry points and access methods
- Available user workflows and interaction patterns
- Context-aware clarifying questions for user
- Recommended UI testing approaches
FOCUS EXCLUSIVELY ON USER INTERFACE - no APIs, databases, or backend analysis.
Output: UI_TEST_DISCOVERY.md ready for user clarification
"""
)
# Present discovery results to user for clarification
print("🖥️ UI Discovery Complete! Please review and clarify your testing objectives:")
print("=" * 60)
display_ui_discovery_results()
print("=" * 60)
# Get user responses to clarification questions
user_responses = collect_user_clarification_responses()
# Generate final UI test objectives based on user input
task_ui_objectives = Task(
subagent_type="ui-test-discovery",
description="Finalize UI test objectives",
prompt=f"""
Create final UI testing objectives based on user responses.
Session Directory: {session_dir}
UI Discovery: {session_dir}/UI_TEST_DISCOVERY.md
User Responses: {user_responses}
Generate UI_TEST_OBJECTIVES.md with:
1. Confirmed UI testing scope and user workflows
2. Specific user personas and contexts for testing
3. Clear success criteria from user experience perspective
4. Testing environment and access requirements
5. Evidence and documentation requirements
Transform user clarifications into actionable UI testing plan.
Output: UI_TEST_OBJECTIVES.md ready for requirements analysis
"""
)
```
**Phase 2: UI Requirements Analysis**
```python
task_requirements = Task(
subagent_type="requirements-analyzer",
description="Extract UI testing requirements from objectives",
prompt=f"""
Transform UI testing objectives into structured testing requirements using markdown communication.
Session Directory: {session_dir}
UI Test Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
Process user-confirmed UI testing objectives:
1. Read UI_TEST_OBJECTIVES.md for user-confirmed testing goals
2. Extract UI-focused acceptance criteria and user workflow requirements
3. Transform user personas and success criteria into testable requirements
4. Identify UI testing dependencies and environment needs
5. Write UI-focused REQUIREMENTS.md to session directory
6. Ensure all requirements focus on user interface and user experience
FOCUS ON USER INTERFACE REQUIREMENTS ONLY - no backend, API, or database requirements.
Output: Complete REQUIREMENTS.md ready for UI scenario generation.
"""
)
task_scenarios = Task(
subagent_type="scenario-designer",
description="Generate UI test scenarios from requirements",
prompt=f"""
Create UI-focused test scenarios using markdown communication.
Session Directory: {session_dir}
Requirements File: {session_dir}/REQUIREMENTS.md
UI Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
Testing Mode: {testing_mode}
Generate comprehensive UI test scenarios:
1. Read REQUIREMENTS.md for UI testing requirements analysis
2. Read UI_TEST_OBJECTIVES.md for user-confirmed workflows and personas
3. Design UI test scenarios covering all user workflows and acceptance criteria
4. Create detailed SCENARIOS.md with step-by-step user interaction procedures
5. Generate BROWSER_INSTRUCTIONS.md with Playwright MCP commands for UI automation
6. Include UI coverage analysis and user workflow traceability
FOCUS EXCLUSIVELY ON USER INTERFACE TESTING - no API, database, or backend scenarios.
Output: SCENARIOS.md and BROWSER_INSTRUCTIONS.md ready for UI test execution.
"""
)
```
**Phase 3: UI Test Execution**
```python
task_ui_browser_execution = Task(
subagent_type="chrome-browser-executor", # MANDATORY: Always use chrome-browser-executor for UI testing
description="Execute automated UI testing with Chrome DevTools",
prompt=f"""
Execute comprehensive UI testing using Chrome DevTools MCP with markdown communication.
Session Directory: {session_dir}
Browser Instructions: {session_dir}/BROWSER_INSTRUCTIONS.md
UI Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
Evidence Directory: {session_dir}/evidence/
Execute all UI test scenarios with user experience focus:
1. Read BROWSER_INSTRUCTIONS.md for detailed UI automation procedures
2. Execute all user workflows using Chrome DevTools MCP tools
3. Capture PNG screenshots of each user interaction step
4. Monitor user interface responsiveness and performance
5. Document user experience issues and accessibility problems
6. Generate comprehensive EXECUTION_LOG.md focused on UI validation
7. Save all evidence in accessible formats for UI analysis
FOCUS ON USER INTERFACE TESTING - validate UI behavior, user workflows, and experience.
Output: Complete EXECUTION_LOG.md with UI testing evidence ready for collection.
"""
)
```
**Phase 4: UI Evidence & Reporting**
```python
task_ui_evidence_collection = Task(
subagent_type="evidence-collector",
description="Collect and organize UI testing evidence",
prompt=f"""
Aggregate UI testing evidence into comprehensive summary using markdown communication.
Session Directory: {session_dir}
Execution Results: {session_dir}/EXECUTION_LOG.md
Evidence Directory: {session_dir}/evidence/
UI Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
Collect and organize UI testing evidence:
1. Read EXECUTION_LOG.md for comprehensive UI test results
2. Catalog all UI evidence files (screenshots, user interaction logs, performance data)
3. Verify evidence accessibility (PNG screenshots, readable formats)
4. Create traceability matrix mapping user workflows to evidence
5. Generate comprehensive EVIDENCE_SUMMARY.md focused on UI validation
FOCUS ON UI TESTING EVIDENCE - user workflows, interface validation, experience assessment.
Output: Complete EVIDENCE_SUMMARY.md ready for UI testing report.
"""
)
task_ui_bmad_reporting = Task(
subagent_type="bmad-reporter",
description="Generate UI testing executive report",
prompt=f"""
Create comprehensive UI testing BMAD report using markdown communication.
Session Directory: {session_dir}
Evidence Summary: {session_dir}/EVIDENCE_SUMMARY.md
UI Objectives: {session_dir}/UI_TEST_OBJECTIVES.md
Requirements Context: {session_dir}/REQUIREMENTS.md
Generate executive UI testing analysis:
1. Read EVIDENCE_SUMMARY.md for comprehensive UI testing evidence
2. Read UI_TEST_OBJECTIVES.md for user-confirmed success criteria
3. Read REQUIREMENTS.md for UI requirements context
4. Synthesize UI testing findings into business impact assessment
5. Develop user experience recommendations with implementation timelines
6. Generate executive BMAD_REPORT.md focused on UI validation results
FOCUS ON USER INTERFACE TESTING OUTCOMES - user experience, UI quality, workflow validation.
Output: Complete BMAD_REPORT.md ready for executive review of UI testing results.
"""
)
```
### Markdown Communication Advantages
#### Enhanced Agent Coordination:
- **Human Readable**: All coordination files in markdown format for easy inspection
- **Standard Templates**: Consistent structure across all testing sessions
- **Accessibility**: Evidence and reports accessible in any text editor or browser
- **Version Control**: All session files can be tracked with git
- **Debugging**: Clear audit trail through markdown file progression
#### Technical Benefits:
- **Simplified Communication**: No complex YAML/JSON parsing required
- **Universal Accessibility**: PNG screenshots viewable in any image software
- **Better Error Recovery**: Markdown files can be manually edited if needed
- **Improved Collaboration**: Human reviewers can validate agent outputs
- **Documentation**: Session becomes self-documenting with markdown files
### Key Framework Improvements
#### Chrome DevTools MCP Integration:
- **Robust Browser Automation**: Direct Chrome DevTools integration for reliable UI testing
- **Enhanced Screenshot Capture**: High-quality PNG screenshots with element-specific capture
- **Performance Monitoring**: Comprehensive network and timing analysis via DevTools
- **Error Handling**: Better failure recovery with detailed error capture
- **Page Management**: Advanced page and tab management capabilities
#### Evidence Management:
- **Accessible Formats**: All evidence in standard, universally accessible formats
- **Organized Storage**: Clear directory structure with descriptive file names
- **Quality Assurance**: Evidence validation and integrity checking
- **Comprehensive Coverage**: Complete traceability from requirements to evidence
### Session Management Features
#### Session Lifecycle Management
```yaml
Session States:
- initialized: Session created, configuration set
- phase_0: Target document loaded and analyzed
- phase_1: Requirements extraction in progress
- phase_2: Test execution in progress
- phase_3: Evidence collection and reporting in progress
- completed: All phases successful, results available
- failed: Unrecoverable error, session terminated
- archived: Session completed and moved to archive
```
#### Cleanup and Maintenance
```yaml
Automatic Cleanup:
- Time-based: Remove sessions > 72 hours old
- Size-based: Archive sessions > 100MB
- Status-based: Remove failed sessions > 24 hours old
- Evidence preservation: Compress successful sessions > 30 days
Manual Cleanup Commands:
- /user_testing --cleanup {session_id}
- /user_testing --cleanup-older-than 7
- /user_testing --archive {session_id}
- /user_testing --list-sessions --include-size
```
#### Error Recovery and Resume
```yaml
Resume Capabilities:
- Checkpoint detection: Identify last successful phase
- State reconstruction: Rebuild session context from files
- Partial retry: Continue from interruption point
- Agent restart: Re-spawn failed agents with existing context
Recovery Procedures:
- Phase 1 failure: Retry requirements extraction
- Phase 2 failure: Switch to manual-only mode if browser automation fails
- Phase 3 failure: Regenerate reports from existing evidence
- Session corruption: Rollback to last successful checkpoint
```
### Integration with Existing Infrastructure
#### Story 3.2 Dependency Integration
```yaml
Prerequisites:
- requirements-analyzer agent: Available and tested
- scenario-designer agent: Available and tested
- validation-planner agent: Available and tested
- Session coordination patterns: Proven in Story 3.2 tests
Integration Pattern:
1. Use existing Story 3.2 agents for phase 1 processing
2. Extend session coordination to phases 2-3
3. Maintain file-based communication compatibility
4. Preserve session schema and validation patterns
```
#### Quality Gates and Validation
```yaml
Quality Gates:
Phase 1 Gates:
- Requirements extraction accuracy ≥ 95%
- Test scenario generation completeness ≥ 90%
- Validation checkpoint coverage = 100%
Phase 2 Gates:
- Test execution completion ≥ 70% scenarios
- Evidence collection success ≥ 90%
- Performance within 5-minute limit
Phase 3 Gates:
- Evidence package validation = 100%
- BMAD report generation = Complete
- Coverage analysis accuracy ≥ 95%
```
### Performance and Monitoring
#### Performance Targets
- **Phase 1**: ≤ 2 minutes for requirements processing
- **Phase 2**: ≤ 5 minutes for test execution
- **Phase 3**: ≤ 1 minute for reporting
- **Total Session**: ≤ 8 minutes for complete epic testing
#### Monitoring and Logging
- Real-time session status updates
- Agent execution progress tracking
- Error detection and alerting
- Performance metrics collection
- Resource usage monitoring
### Command Output
#### Success Output
```
✅ BMAD Testing Session Completed Successfully
Session ID: epic-3.3_hybrid_20250829_143000_abc123
Target: Epic 3.3 - Test Execution & BMAD Reporting Engine
Mode: Hybrid (Automated + Manual)
Duration: 4.2 minutes
📊 Results Summary:
- Acceptance Criteria Coverage: 85.7% (6/7 ACs)
- Test Scenarios Executed: 12/15
- Evidence Files Generated: 41
- Issues Found: 2 Major, 3 Minor
- Recommendations: 8 actionable items
📋 Reports Generated:
- BMAD Brief: workspace/testing/sessions/{session_id}/phase_3/bmad_brief.md
- Recommendations: workspace/testing/sessions/{session_id}/phase_3/recommendations.json
- Evidence Package: workspace/testing/sessions/{session_id}/phase_2/evidence/package.json
🎯 Next Steps:
1. Review BMAD brief for critical findings
2. Implement high-priority recommendations
3. Address browser automation reliability issues
Session archived to: workspace/testing/archive/2025-08-29/
```
#### Error Output
```
❌ BMAD Testing Session Failed
Session ID: epic-3.3_hybrid_20250829_143000_abc123
Target: Epic 3.3 - Test Execution & BMAD Reporting Engine
Duration: 2.1 minutes (failed in Phase 2)
🔍 Failure Analysis:
- Phase 1: ✅ Completed successfully
- Phase 2: ❌ Browser automation timeout, manual testing incomplete
- Phase 3: ⏸️ Not reached
🛠️ Recovery Options:
1. Retry with interactive-only mode: /user_testing epic-3.3 --mode interactive
2. Resume from Phase 2: /user_testing --resume epic-3.3_hybrid_20250829_143000_abc123
3. Review detailed logs: workspace/testing/sessions/{session_id}/phase_2/execution_log.json
### Browser Session Troubleshooting
If tests fail with "Browser is already in use" error:
1. **Close Chrome windows**: Look for Chrome DevTools-opened Chrome windows and close them
2. **Check page status**: Use Chrome DevTools list_pages to see active sessions
3. **Retry test**: Browser session will be available for next test
Session preserved for debugging. Use --cleanup to remove when resolved.
```
---
*This command orchestrates the complete BMAD testing workflow through Claude-native Task tool coordination, providing comprehensive epic testing with structured reporting in under 8 minutes.*
View Git Blame Copy Permalink