BMAD-METHOD/expansion-packs/bmad-playwright-expansion/tasks/validate-scenarios.md

Validate Scenarios

Purpose

Validate natural language test scenarios for a user story using Playwright MCP through interactive browser testing. This command validates scenarios generated by *generate-scenarios and discovers fixes through real browser interaction.

Prerequisites

• CRITICAL: User must have successfully executed *generate-scenarios command first
• Natural language test scenarios must be available from the previous generation phase
• Playwright MCP server is installed and configured
• Frontend and backend applications are running locally
• Test database is set up and accessible
• OpenAPI documentation is available

Dependency Validation

Before executing this task, verify:

1. Scenario Generation Status: Confirm *generate-scenarios was completed successfully
2. Scenario Availability: Natural language scenarios are available for validation
3. Environment Setup: All required services are running
4. Playwright MCP: Server is accessible and configured

Inputs Required

1. Story Identifier: Which story's scenarios to validate (e.g., "Story 2.1", "docs/stories/2.1.story.md")
2. Environment URLs:
   • Frontend URL (default: http://localhost:3000)
   • Backend URL (default: http://localhost:8000)
   • OpenAPI URL (default: http://localhost:8000/swagger/)
3. Validation Scope: Full comprehensive validation or specific areas (API, E2E, Security, etc.)
4. Browser Coverage: Which browsers to test (default: chromium, firefox, webkit)

Process

Phase 1: Scenario Loading & Environment Preparation

1. *Load generated scenarios from previous generate-scenarios execution

   • Read natural language test scenarios
   • Parse scenario structure and requirements
   • Identify all test cases and edge cases
   • Note security requirements and authentication needs

2. Verify test environment is ready

   Use Playwright_navigate to navigate to {frontend_url} and verify application loads
   Use Playwright_get to call {backend_url}/api/health/ and verify API is responding
   Use Playwright_navigate to {openapi_url} and verify API documentation is accessible
   

3. Set up test data if needed

   • Create test users with appropriate roles
   • Set up any required test data in database
   • Prepare authentication tokens for API testing

Phase 2: Authentication & Authorization Validation

1. Validate unauthenticated access scenarios

   Use Playwright_navigate to navigate to {frontend_url}
   Attempt to access protected routes without authentication
   Verify redirects to login page occur
   Use Playwright_get to test API endpoints without authentication tokens
   Verify 401 Unauthorized responses
   Use Playwright_console_logs to check for any authorization errors
   

2. Validate user registration and login flow scenarios

   Use Playwright_navigate to navigate to {frontend_url}/register
   Use Playwright_fill to fill registration form with test data
   Use Playwright_click to submit registration and verify success
   Use Playwright_navigate to navigate to login page
   Use Playwright_fill to fill login form with created credentials
   Use Playwright_click to submit login form
   Use Playwright_console_logs to verify JWT tokens are stored
   Use Playwright_get_visible_text to verify redirect to appropriate dashboard
   

3. Validate role-based access control scenarios

   Login as different user types (vendor, admin, etc.)
   Use Playwright_navigate to verify each role can only access appropriate features
   Use Playwright_get to test API endpoints with different user roles
   Verify proper authorization responses (403 for unauthorized)
   Use Playwright_get to test row-level security (users can only see their own data)
   Use Playwright_console_logs to check for any authorization errors
   

Phase 3: API Validation

1. Validate all API endpoint scenarios

   For each endpoint in the OpenAPI documentation:
   - Use Playwright_get to test without authentication (expect 401)
   - Use Playwright_get to test with invalid token (expect 401)
   - Use Playwright_get to test with valid token (expect appropriate response)
   - Use Playwright_post/put/patch/delete for CRUD operations if applicable
   - Use Playwright_post to test input validation (invalid data should return 400)
   - Test edge cases and boundary conditions
   

2. Validate API security scenarios

   Use Playwright_post to test for SQL injection vulnerabilities
   Use Playwright_post to test for XSS vulnerabilities in API responses
   Verify proper input sanitization
   Use Playwright_get to test rate limiting on API endpoints
   Verify CSRF protection on state-changing operations
   Use Playwright_console_logs to monitor for security warnings
   

Phase 4: Frontend E2E Validation

1. Validate complete user journey scenarios

   Execute the primary user flow described in the story
   Use Playwright_navigate through each step of the acceptance criteria
   Use Playwright_click, Playwright_fill, Playwright_select for interactions
   Use Playwright_screenshot to take screenshots at key steps for documentation
   Use Playwright_get_visible_text to verify each interaction works as expected
   Use Playwright_console_logs to check for JavaScript errors
   Verify final state matches expected outcome
   

2. Validate form interaction scenarios

   For each form in the story:
   - Use Playwright_fill to test form validation (empty fields, invalid data)
   - Use Playwright_click to test successful form submission
   - Use Playwright_get_visible_text to verify error messages are user-friendly
   - Use Playwright_press_key to test keyboard navigation and accessibility
   

3. Validate responsive design scenarios

   Use Playwright_navigate with width=1920 and height=1080 to test desktop viewport
   Use Playwright_screenshot to capture desktop view
   Use Playwright_navigate with width=768 and height=1024 to test tablet viewport
   Use Playwright_screenshot to capture tablet view
   Use Playwright_navigate with width=375 and height=667 to test mobile viewport
   Use Playwright_screenshot to capture mobile view
   Verify layout adapts correctly by comparing screenshots
   

Phase 5: Integration Validation

1. Validate frontend-backend integration scenarios

   Use Playwright_expect_response to set up monitoring for API calls
   Use Playwright_click to trigger actions that make API calls
   Use Playwright_assert_response to verify API calls are made correctly
   Use Playwright_console_logs to verify no network errors occurred
   Test error handling for API failures
   Test loading states during API calls
   

2. Validate database integration scenarios

   Use Playwright_navigate to create data via frontend
   Use Playwright_get to verify data exists in database via API
   Use Playwright_navigate to modify data via frontend
   Use Playwright_get to verify changes persist in database
   Use Playwright_navigate to delete data via frontend
   Use Playwright_get to verify removal from database
   

Phase 6: Cross-Browser Validation

1. Validate critical flows in multiple browsers

   For each browser (chromium, firefox, webkit):
   - Use Playwright_navigate with browserType to execute primary user journey
   - Test authentication flows
   - Verify JavaScript functionality
   - Test form submissions
   - Use Playwright_screenshot to verify responsive design renders correctly
   - Use Playwright_console_logs to check for browser-specific issues
   

Phase 7: Security Audit Validation

1. Validate comprehensive security scenarios

   Use Playwright_post to test password security requirements
   Verify secure token storage (httpOnly cookies)
   Use Playwright_post to test CSRF protection
   Use Playwright_get to test rate limiting
   Use Playwright_get to verify data isolation between users
   Use Playwright_post to test input sanitization and output escaping
   Use Playwright_console_logs to monitor for security warnings
   

Phase 8: Performance Validation

1. Validate page load time scenarios

   Use Playwright_navigate to navigate to key pages and measure load times
   Use Playwright_console_logs to check for performance warnings
   Use Playwright_screenshot to document page states
   Test with simulated slow network conditions
   

2. Validate API performance scenarios

   Use Playwright_get to measure API response times
   Use Playwright_evaluate to test with concurrent requests
   Use Playwright_console_logs to monitor for performance warnings
   Verify database query optimization
   

Phase 9: Error Handling Validation

1. Validate network error scenarios

   Use Playwright_evaluate to simulate network failures during operations
   Use Playwright_get_visible_text to verify graceful error handling
   Test retry mechanisms
   Use Playwright_console_logs to verify user feedback for errors
   

2. Validate server error scenarios

   Use Playwright_evaluate to simulate 500 Internal Server Error responses
   Use Playwright_get_visible_text to verify user-friendly error messages
   Use Playwright_console_logs to verify application doesn't crash
   

Output

Validation Results Summary

• Total Scenarios Validated: {count}
• Scenarios Passed: {count}
• Scenarios Failed: {count}
• Issues Found and Fixed: {count}
• Security Issues Found: {count}
• Performance Issues: {count}
• Cross-Browser Issues: {count}

Detailed Validation Report

• Authentication & Authorization: {status and details}
• API Validation: {status and details}
• Frontend E2E: {status and details}
• Integration Validation: {status and details}
• Security Audit: {status and details}
• Performance Validation: {status and details}
• Cross-Browser Compatibility: {status and details}

Issues Found and Fixes Applied

• List any bugs, security vulnerabilities, or performance issues discovered
• Include screenshots and console logs where relevant
• Document fixes and adjustments made during validation
• Provide recommendations for improvements

Validated Scenarios Ready for Code Generation

• All scenarios that passed validation
• Fixes and adjustments incorporated
• Context for *generate-test-files command

Success Criteria

• All acceptance criteria from the story are validated through browser interaction
• No critical security vulnerabilities found
• All API endpoints function correctly with proper authentication
• User journeys work across all supported browsers
• Performance meets established benchmarks
• Error scenarios are handled gracefully
• Data integrity is maintained throughout all operations
• All scenarios are ready for conversion to TypeScript test files

Notes

• This task validates scenarios generated by *generate-scenarios
• All validation is done through interactive Playwright MCP browser testing
• Issues discovered are fixed in real-time during validation
• Results provide context and fixes for *generate-test-files command
• Validated scenarios serve as the foundation for production test code generation