BMAD-METHOD/src/bmm/workflows/5-validation/uat-validate/steps/step-02-classify-scenarios.md

Step 2: Classify Scenarios

Purpose

Categorize each scenario by its executability level to determine which can be automated, which need partial automation, and which require manual verification.

Inputs

Input	Source	Required
scenario_list	Step 1	Yes

Process

2.1 Classification Categories

Classification	Description	Gate Behavior
Automatable	Can be fully executed via shell command	Execute and verify
Semi-automated	Requires setup, then automated verification	Execute with warning
Manual	Requires human interaction or visual verification	Skip, add to checklist

2.2 Detect Automatable Scenarios

Check scenario content for these indicators (case-insensitive):

CLI/Command indicators:

• npx, npm run, yarn, pnpm
• node , python , ruby
• curl , wget , http
• bash , sh , ./

Test framework indicators:

• pytest, jest, vitest, mocha
• npm test, yarn test

Verification indicators:

• --version, --help
• /health, /api/, /status
• exit code, returns 0, returns 1
• outputs, prints, displays

Database/Config indicators:

• db migrate, db status
• config validate, config check

2.3 Detect Semi-Automated Scenarios

Scenarios with commands that require prior setup:

Setup-required indicators:

• "Start the server first"
• "Ensure database is running"
• "In a separate terminal"
• "After deploying"

Partial automation indicators:

• test-send, send test
• "check your email/inbox"
• "verify in browser"
• Manual setup + automated verification

2.4 Classify as Manual

Scenarios without detectable automation path:

Manual-only indicators:

• "Railway dashboard", "Vercel dashboard"
• "Open browser", "Navigate to"
• "Visual inspection", "Visually verify"
• "Two terminals", "Side by side"
• "User should see", "Observe that"
• No code blocks or CLI references

2.5 Extract Commands

For automatable/semi-automated scenarios, extract the verification command:

1. Look for inline code: `command here`
2. Look for code blocks: bash ...
3. Look for CLI patterns: npx ..., npm run ..., curl ...
4. Look for expected patterns after "Run:" or "Execute:"

2.6 Build Classification Result

classification:
  total: 9
  automatable: 6
  semi_automated: 2
  manual: 1

automatable_scenarios:
  - id: 1
    name: "Project Initialization"
    command: "npx heimdall --version"
    expected: "displays a version number"

  - id: 3
    name: "Database Migration"
    command: "npx heimdall db migrate"
    expected: "success message"

semi_automated_scenarios:
  - id: 7
    name: "Email Notification"
    command: "curl -X POST localhost:3000/test-send"
    expected: "email received"
    note: "Requires manual inbox verification"

manual_scenarios:
  - id: 9
    name: "Dashboard Visual Check"
    note: "Requires browser inspection"

Outputs

Output	Location	Description
automatable	Array	Scenarios to execute automatically
semi_automated	Array	Scenarios needing setup + execution
manual	Array	Scenarios requiring human verification
classification_summary	Console	Counts per category

Completion Signal

SCENARIOS_CLASSIFIED: {automatable}/{semi_automated}/{manual}

Example Output

[UAT] Classifying 9 scenarios...
  [AUTO] Scenario 1: Project Initialization
  [AUTO] Scenario 2: Configuration Setup
  [AUTO] Scenario 3: Database Migration
  [AUTO] Scenario 4: Connection Validation
  [AUTO] Scenario 5: Worker Process Startup
  [AUTO] Scenario 6: Job Queue Testing
  [SEMI] Scenario 7: Email Notification
  [SEMI] Scenario 8: Webhook Delivery
  [MANUAL] Scenario 9: Dashboard Visual Check

SCENARIOS_CLASSIFIED: 6/2/1

Classification Confidence

For edge cases, use this priority:

1. If command is clearly present → Automatable
2. If command present but requires setup → Semi-automated
3. If no command detected → Manual

When in doubt, classify as semi-automated (attempts execution, flags for review).