240 lines
6.8 KiB
Markdown
240 lines
6.8 KiB
Markdown
# Test Coverage Requirements Task
|
|
|
|
## Purpose
|
|
Define and enforce comprehensive test coverage requirements to ensure code quality, prevent regressions, and maintain system reliability. This task establishes testing standards and validates compliance across all test levels.
|
|
|
|
## Integration with Memory System
|
|
- **What patterns to search for**: Test coverage trends, common test gaps, regression patterns, test maintenance burden
|
|
- **What outcomes to track**: Coverage percentages, test execution times, defect escape rates, regression frequency
|
|
- **What learnings to capture**: Effective test strategies, high-value test areas, test automation ROI, maintenance patterns
|
|
|
|
## Test Coverage Categories
|
|
|
|
### Unit Test Requirements
|
|
```yaml
|
|
unit_test_coverage:
|
|
minimum_coverage: 90%
|
|
critical_paths: 100%
|
|
|
|
required_tests:
|
|
- happy_path: All success scenarios
|
|
- edge_cases: Boundary conditions
|
|
- error_handling: Exception scenarios
|
|
- null_checks: Null/undefined inputs
|
|
- validation: Input validation logic
|
|
|
|
excluded_from_coverage:
|
|
- generated_code: Auto-generated files
|
|
- config_files: Static configurations
|
|
- type_definitions: Interface/type files
|
|
```
|
|
|
|
### Integration Test Requirements
|
|
- [ ] **API Tests**: All endpoints with various payloads
|
|
- [ ] **Database Tests**: CRUD operations, transactions
|
|
- [ ] **External Service Tests**: Mock integrations
|
|
- [ ] **Message Queue Tests**: Pub/sub scenarios
|
|
- [ ] **Authentication Tests**: Auth flows, permissions
|
|
|
|
### End-to-End Test Requirements
|
|
- [ ] **Critical User Journeys**: Primary workflows
|
|
- [ ] **Cross-Browser Tests**: Major browser support
|
|
- [ ] **Performance Tests**: Load time requirements
|
|
- [ ] **Accessibility Tests**: WCAG compliance
|
|
- [ ] **Mobile Tests**: Responsive behavior
|
|
|
|
## Coverage Measurement Framework
|
|
|
|
### Step 1: Coverage Analysis
|
|
```python
|
|
def analyze_test_coverage():
|
|
coverage_report = {
|
|
"unit": {
|
|
"line_coverage": calculate_line_coverage(),
|
|
"branch_coverage": calculate_branch_coverage(),
|
|
"function_coverage": calculate_function_coverage(),
|
|
"statement_coverage": calculate_statement_coverage()
|
|
},
|
|
"integration": {
|
|
"api_coverage": calculate_api_endpoint_coverage(),
|
|
"scenario_coverage": calculate_business_scenario_coverage(),
|
|
"error_coverage": calculate_error_scenario_coverage()
|
|
},
|
|
"e2e": {
|
|
"user_journey_coverage": calculate_journey_coverage(),
|
|
"browser_coverage": calculate_browser_coverage(),
|
|
"device_coverage": calculate_device_coverage()
|
|
}
|
|
}
|
|
return coverage_report
|
|
```
|
|
|
|
### Step 2: Gap Identification
|
|
```markdown
|
|
## Test Coverage Gap Analysis
|
|
**Component**: {component_name}
|
|
**Current Coverage**: {current}%
|
|
**Required Coverage**: {required}%
|
|
**Gap**: {gap}%
|
|
|
|
### Uncovered Areas
|
|
1. **{Area}**: {description}
|
|
- Risk Level: {high/medium/low}
|
|
- Priority: {priority}
|
|
- Estimated Effort: {effort}
|
|
|
|
### Recommended Tests
|
|
- {test_type}: {test_description}
|
|
```
|
|
|
|
### Step 3: Test Quality Validation
|
|
| Test Aspect | Requirement | Status | Notes |
|
|
|-------------|-------------|---------|--------|
|
|
| Assertions | Meaningful assertions | ✓/✗ | {notes} |
|
|
| Independence | No test interdependence | ✓/✗ | {notes} |
|
|
| Repeatability | Consistent results | ✓/✗ | {notes} |
|
|
| Performance | <2s for unit tests | ✓/✗ | {notes} |
|
|
| Clarity | Self-documenting | ✓/✗ | {notes} |
|
|
|
|
## Quality Gates
|
|
|
|
### Development Gate
|
|
- [ ] Unit tests written for new code
|
|
- [ ] Coverage threshold maintained
|
|
- [ ] All tests passing locally
|
|
- [ ] No skipped tests without justification
|
|
|
|
### Pull Request Gate
|
|
- [ ] Coverage report generated
|
|
- [ ] No coverage decrease
|
|
- [ ] Integration tests updated
|
|
- [ ] Test documentation current
|
|
|
|
### Release Gate
|
|
- [ ] E2E tests passing
|
|
- [ ] Performance benchmarks met
|
|
- [ ] Security tests passing
|
|
- [ ] Regression suite complete
|
|
|
|
## Test Strategy Guidelines
|
|
|
|
### Test Pyramid Balance
|
|
```python
|
|
test_distribution = {
|
|
"unit_tests": {
|
|
"percentage": 70,
|
|
"execution_time": "< 5 minutes",
|
|
"frequency": "every commit"
|
|
},
|
|
"integration_tests": {
|
|
"percentage": 20,
|
|
"execution_time": "< 15 minutes",
|
|
"frequency": "every PR"
|
|
},
|
|
"e2e_tests": {
|
|
"percentage": 10,
|
|
"execution_time": "< 30 minutes",
|
|
"frequency": "before release"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Critical Path Identification
|
|
```python
|
|
critical_paths = [
|
|
"user_authentication_flow",
|
|
"payment_processing",
|
|
"data_integrity_operations",
|
|
"security_validations",
|
|
"core_business_logic"
|
|
]
|
|
|
|
# These paths require 100% coverage
|
|
```
|
|
|
|
## Success Criteria
|
|
- Overall test coverage >90%
|
|
- Critical path coverage 100%
|
|
- Zero untested public methods
|
|
- Test execution time within limits
|
|
- Defect escape rate <5%
|
|
|
|
## Memory Integration
|
|
```python
|
|
# Test coverage memory
|
|
test_coverage_memory = {
|
|
"type": "test_coverage_analysis",
|
|
"snapshot": {
|
|
"timestamp": analysis_time,
|
|
"project": project_name,
|
|
"version": code_version
|
|
},
|
|
"coverage": {
|
|
"unit": unit_coverage_details,
|
|
"integration": integration_coverage_details,
|
|
"e2e": e2e_coverage_details,
|
|
"overall": weighted_average
|
|
},
|
|
"gaps": {
|
|
"identified": coverage_gaps,
|
|
"risk_assessment": gap_risks,
|
|
"remediation_plan": improvement_plan
|
|
},
|
|
"trends": {
|
|
"coverage_trend": historical_comparison,
|
|
"test_growth": test_count_trend,
|
|
"execution_time": performance_trend
|
|
},
|
|
"quality": {
|
|
"flaky_tests": unstable_test_count,
|
|
"slow_tests": performance_outliers,
|
|
"skipped_tests": disabled_test_count
|
|
}
|
|
}
|
|
```
|
|
|
|
## Coverage Report Template
|
|
```markdown
|
|
# Test Coverage Report
|
|
**Project**: {project_name}
|
|
**Date**: {timestamp}
|
|
**Overall Coverage**: {percentage}%
|
|
|
|
## Coverage Summary
|
|
| Type | Required | Actual | Gap | Status |
|
|
|------|----------|--------|-----|---------|
|
|
| Unit | 90% | {n}% | {g}% | {✓/✗} |
|
|
| Integration | 80% | {n}% | {g}% | {✓/✗} |
|
|
| E2E | 70% | {n}% | {g}% | {✓/✗} |
|
|
|
|
## Critical Path Coverage
|
|
| Path | Coverage | Tests | Status |
|
|
|------|----------|--------|--------|
|
|
| {path} | {cov}% | {count} | {status} |
|
|
|
|
## Test Quality Metrics
|
|
- **Total Tests**: {count}
|
|
- **Execution Time**: {time}
|
|
- **Flaky Tests**: {count}
|
|
- **Skipped Tests**: {count}
|
|
|
|
## Coverage Gaps - High Priority
|
|
1. **{Component}**: {current}% → {target}%
|
|
- Missing: {test_types}
|
|
- Risk: {risk_level}
|
|
- Action: {action_plan}
|
|
|
|
## Recommendations
|
|
1. **Immediate**: {urgent_gaps}
|
|
2. **Next Sprint**: {planned_improvements}
|
|
3. **Long-term**: {strategic_improvements}
|
|
|
|
## Test Maintenance Needs
|
|
{test_refactoring_requirements}
|
|
```
|
|
|
|
## Brotherhood Collaboration
|
|
- Coverage review with development team
|
|
- Test strategy with QA team
|
|
- Risk assessment with product team
|
|
- Performance impact with DevOps team |