BMAD-METHOD/expansion-packs/bmad-playwright-expansion/tasks/validate-scenarios.md

# Validate Scenarios

## Purpose
Validate natural language test scenarios for a user story using Playwright MCP through interactive browser testing. This command validates scenarios generated by *generate-scenarios and discovers fixes through real browser interaction.

## Prerequisites
- **CRITICAL**: User must have successfully executed `*generate-scenarios` command first
- Natural language test scenarios must be available from the previous generation phase
- Playwright MCP server is installed and configured
- Frontend and backend applications are running locally
- Test database is set up and accessible
- OpenAPI documentation is available

## Dependency Validation
Before executing this task, verify:
1. **Scenario Generation Status**: Confirm `*generate-scenarios` was completed successfully
2. **Scenario Availability**: Natural language scenarios are available for validation
3. **Environment Setup**: All required services are running
4. **Playwright MCP**: Server is accessible and configured

## Inputs Required
1. **Story Identifier**: Which story's scenarios to validate (e.g., "Story 2.1", "docs/stories/2.1.story.md")
2. **Environment URLs**: 
   - Frontend URL (default: http://localhost:3000)
   - Backend URL (default: http://localhost:8000)
   - OpenAPI URL (default: http://localhost:8000/swagger/)
3. **Validation Scope**: Full comprehensive validation or specific areas (API, E2E, Security, etc.)
4. **Browser Coverage**: Which browsers to test (default: chromium, firefox, webkit)

## Process

### Phase 1: Scenario Loading & Environment Preparation
1. **Load generated scenarios from previous *generate-scenarios execution**
   - Read natural language test scenarios
   - Parse scenario structure and requirements
   - Identify all test cases and edge cases
   - Note security requirements and authentication needs

2. **Verify test environment is ready**
   ```
   Use Playwright_navigate to navigate to {frontend_url} and verify application loads
   Use Playwright_get to call {backend_url}/api/health/ and verify API is responding
   Use Playwright_navigate to {openapi_url} and verify API documentation is accessible
   ```

3. **Set up test data if needed**
   - Create test users with appropriate roles
   - Set up any required test data in database
   - Prepare authentication tokens for API testing

### Phase 2: Authentication & Authorization Validation
1. **Validate unauthenticated access scenarios**
   ```
   Use Playwright_navigate to navigate to {frontend_url}
   Attempt to access protected routes without authentication
   Verify redirects to login page occur
   Use Playwright_get to test API endpoints without authentication tokens
   Verify 401 Unauthorized responses
   Use Playwright_console_logs to check for any authorization errors
   ```

2. **Validate user registration and login flow scenarios**
   ```
   Use Playwright_navigate to navigate to {frontend_url}/register
   Use Playwright_fill to fill registration form with test data
   Use Playwright_click to submit registration and verify success
   Use Playwright_navigate to navigate to login page
   Use Playwright_fill to fill login form with created credentials
   Use Playwright_click to submit login form
   Use Playwright_console_logs to verify JWT tokens are stored
   Use Playwright_get_visible_text to verify redirect to appropriate dashboard
   ```

3. **Validate role-based access control scenarios**
   ```
   Login as different user types (vendor, admin, etc.)
   Use Playwright_navigate to verify each role can only access appropriate features
   Use Playwright_get to test API endpoints with different user roles
   Verify proper authorization responses (403 for unauthorized)
   Use Playwright_get to test row-level security (users can only see their own data)
   Use Playwright_console_logs to check for any authorization errors
   ```

### Phase 3: API Validation
1. **Validate all API endpoint scenarios**
   ```
   For each endpoint in the OpenAPI documentation:
   - Use Playwright_get to test without authentication (expect 401)
   - Use Playwright_get to test with invalid token (expect 401)
   - Use Playwright_get to test with valid token (expect appropriate response)
   - Use Playwright_post/put/patch/delete for CRUD operations if applicable
   - Use Playwright_post to test input validation (invalid data should return 400)
   - Test edge cases and boundary conditions
   ```

2. **Validate API security scenarios**
   ```
   Use Playwright_post to test for SQL injection vulnerabilities
   Use Playwright_post to test for XSS vulnerabilities in API responses
   Verify proper input sanitization
   Use Playwright_get to test rate limiting on API endpoints
   Verify CSRF protection on state-changing operations
   Use Playwright_console_logs to monitor for security warnings
   ```

### Phase 4: Frontend E2E Validation
1. **Validate complete user journey scenarios**
   ```
   Execute the primary user flow described in the story
   Use Playwright_navigate through each step of the acceptance criteria
   Use Playwright_click, Playwright_fill, Playwright_select for interactions
   Use Playwright_screenshot to take screenshots at key steps for documentation
   Use Playwright_get_visible_text to verify each interaction works as expected
   Use Playwright_console_logs to check for JavaScript errors
   Verify final state matches expected outcome
   ```

2. **Validate form interaction scenarios**
   ```
   For each form in the story:
   - Use Playwright_fill to test form validation (empty fields, invalid data)
   - Use Playwright_click to test successful form submission
   - Use Playwright_get_visible_text to verify error messages are user-friendly
   - Use Playwright_press_key to test keyboard navigation and accessibility
   ```

3. **Validate responsive design scenarios**
   ```
   Use Playwright_navigate with width=1920 and height=1080 to test desktop viewport
   Use Playwright_screenshot to capture desktop view
   Use Playwright_navigate with width=768 and height=1024 to test tablet viewport
   Use Playwright_screenshot to capture tablet view
   Use Playwright_navigate with width=375 and height=667 to test mobile viewport
   Use Playwright_screenshot to capture mobile view
   Verify layout adapts correctly by comparing screenshots
   ```

### Phase 5: Integration Validation
1. **Validate frontend-backend integration scenarios**
   ```
   Use Playwright_expect_response to set up monitoring for API calls
   Use Playwright_click to trigger actions that make API calls
   Use Playwright_assert_response to verify API calls are made correctly
   Use Playwright_console_logs to verify no network errors occurred
   Test error handling for API failures
   Test loading states during API calls
   ```

2. **Validate database integration scenarios**
   ```
   Use Playwright_navigate to create data via frontend
   Use Playwright_get to verify data exists in database via API
   Use Playwright_navigate to modify data via frontend
   Use Playwright_get to verify changes persist in database
   Use Playwright_navigate to delete data via frontend
   Use Playwright_get to verify removal from database
   ```

### Phase 6: Cross-Browser Validation
1. **Validate critical flows in multiple browsers**
   ```
   For each browser (chromium, firefox, webkit):
   - Use Playwright_navigate with browserType to execute primary user journey
   - Test authentication flows
   - Verify JavaScript functionality
   - Test form submissions
   - Use Playwright_screenshot to verify responsive design renders correctly
   - Use Playwright_console_logs to check for browser-specific issues
   ```

### Phase 7: Security Audit Validation
1. **Validate comprehensive security scenarios**
   ```
   Use Playwright_post to test password security requirements
   Verify secure token storage (httpOnly cookies)
   Use Playwright_post to test CSRF protection
   Use Playwright_get to test rate limiting
   Use Playwright_get to verify data isolation between users
   Use Playwright_post to test input sanitization and output escaping
   Use Playwright_console_logs to monitor for security warnings
   ```

### Phase 8: Performance Validation
1. **Validate page load time scenarios**
   ```
   Use Playwright_navigate to navigate to key pages and measure load times
   Use Playwright_console_logs to check for performance warnings
   Use Playwright_screenshot to document page states
   Test with simulated slow network conditions
   ```

2. **Validate API performance scenarios**
   ```
   Use Playwright_get to measure API response times
   Use Playwright_evaluate to test with concurrent requests
   Use Playwright_console_logs to monitor for performance warnings
   Verify database query optimization
   ```

### Phase 9: Error Handling Validation
1. **Validate network error scenarios**
   ```
   Use Playwright_evaluate to simulate network failures during operations
   Use Playwright_get_visible_text to verify graceful error handling
   Test retry mechanisms
   Use Playwright_console_logs to verify user feedback for errors
   ```

2. **Validate server error scenarios**
   ```
   Use Playwright_evaluate to simulate 500 Internal Server Error responses
   Use Playwright_get_visible_text to verify user-friendly error messages
   Use Playwright_console_logs to verify application doesn't crash
   ```

## Output

### Validation Results Summary
- **Total Scenarios Validated**: {count}
- **Scenarios Passed**: {count}
- **Scenarios Failed**: {count}
- **Issues Found and Fixed**: {count}
- **Security Issues Found**: {count}
- **Performance Issues**: {count}
- **Cross-Browser Issues**: {count}

### Detailed Validation Report
- **Authentication & Authorization**: {status and details}
- **API Validation**: {status and details}
- **Frontend E2E**: {status and details}
- **Integration Validation**: {status and details}
- **Security Audit**: {status and details}
- **Performance Validation**: {status and details}
- **Cross-Browser Compatibility**: {status and details}

### Issues Found and Fixes Applied
- List any bugs, security vulnerabilities, or performance issues discovered
- Include screenshots and console logs where relevant
- Document fixes and adjustments made during validation
- Provide recommendations for improvements

### Validated Scenarios Ready for Code Generation
- All scenarios that passed validation
- Fixes and adjustments incorporated
- Context for *generate-test-files command

## Success Criteria
- All acceptance criteria from the story are validated through browser interaction
- No critical security vulnerabilities found
- All API endpoints function correctly with proper authentication
- User journeys work across all supported browsers
- Performance meets established benchmarks
- Error scenarios are handled gracefully
- Data integrity is maintained throughout all operations
- All scenarios are ready for conversion to TypeScript test files

## Notes
- This task validates scenarios generated by *generate-scenarios
- All validation is done through interactive Playwright MCP browser testing
- Issues discovered are fixed in real-time during validation
- Results provide context and fixes for *generate-test-files command
- Validated scenarios serve as the foundation for production test code generation