268 lines
8.1 KiB
Markdown
268 lines
8.1 KiB
Markdown
# Quality Metrics Tracking Task
|
|
|
|
## Purpose
|
|
Define, collect, analyze, and track comprehensive quality metrics across all development activities. This task establishes a data-driven approach to quality improvement and provides visibility into quality trends and patterns.
|
|
|
|
## Integration with Memory System
|
|
- **What patterns to search for**: Metric trend patterns, quality improvement correlations, threshold violations, anomaly patterns
|
|
- **What outcomes to track**: Quality improvement rates, metric stability, alert effectiveness, action item completion
|
|
- **What learnings to capture**: Effective metric thresholds, leading indicators, improvement strategies, metric correlations
|
|
|
|
## Quality Metrics Categories
|
|
|
|
### Code Quality Metrics
|
|
```yaml
|
|
code_quality_metrics:
|
|
static_analysis:
|
|
- complexity: Cyclomatic complexity per function
|
|
- duplication: Code duplication percentage
|
|
- maintainability: Maintainability index
|
|
- technical_debt: Debt ratio and time
|
|
|
|
dynamic_analysis:
|
|
- test_coverage: Line, branch, function coverage
|
|
- mutation_score: Test effectiveness
|
|
- performance: Response times, resource usage
|
|
- reliability: Error rates, crash frequency
|
|
```
|
|
|
|
### Process Quality Metrics
|
|
- [ ] **Development Velocity**: Story points completed
|
|
- [ ] **Defect Density**: Defects per KLOC
|
|
- [ ] **Lead Time**: Idea to production time
|
|
- [ ] **Cycle Time**: Development start to done
|
|
- [ ] **Review Efficiency**: Review time and effectiveness
|
|
|
|
### Product Quality Metrics
|
|
- [ ] **User Satisfaction**: NPS, CSAT scores
|
|
- [ ] **Defect Escape Rate**: Production bugs
|
|
- [ ] **Mean Time to Recovery**: Incident resolution
|
|
- [ ] **Feature Adoption**: Usage analytics
|
|
- [ ] **Performance SLAs**: Uptime, response times
|
|
|
|
## Metric Collection Framework
|
|
|
|
### Step 1: Automated Collection
|
|
```python
|
|
def collect_quality_metrics():
|
|
metrics = {
|
|
"code": {
|
|
"coverage": get_test_coverage(),
|
|
"complexity": calculate_complexity(),
|
|
"duplication": detect_duplication(),
|
|
"violations": count_lint_violations()
|
|
},
|
|
"process": {
|
|
"velocity": calculate_velocity(),
|
|
"lead_time": measure_lead_time(),
|
|
"review_time": average_review_time(),
|
|
"build_success": build_success_rate()
|
|
},
|
|
"product": {
|
|
"availability": calculate_uptime(),
|
|
"performance": measure_response_times(),
|
|
"errors": count_error_rates(),
|
|
"satisfaction": get_user_scores()
|
|
}
|
|
}
|
|
return enrich_with_trends(metrics)
|
|
```
|
|
|
|
### Step 2: Metric Analysis
|
|
```python
|
|
def analyze_metrics(current_metrics, historical_data):
|
|
analysis = {
|
|
"trends": calculate_trends(current_metrics, historical_data),
|
|
"anomalies": detect_anomalies(current_metrics),
|
|
"correlations": find_correlations(current_metrics),
|
|
"predictions": forecast_trends(historical_data),
|
|
"health_score": calculate_overall_health(current_metrics)
|
|
}
|
|
|
|
return generate_insights(analysis)
|
|
```
|
|
|
|
### Step 3: Threshold Management
|
|
| Metric | Green | Yellow | Red | Action |
|
|
|--------|-------|---------|-----|---------|
|
|
| Test Coverage | >90% | 80-90% | <80% | Block deployment |
|
|
| Complexity | <10 | 10-20 | >20 | Refactor required |
|
|
| Build Success | >95% | 85-95% | <85% | Fix immediately |
|
|
| Review Time | <4hr | 4-8hr | >8hr | Escalate |
|
|
| Error Rate | <0.1% | 0.1-1% | >1% | Incident response |
|
|
|
|
## Quality Dashboard Design
|
|
|
|
### Real-Time Metrics
|
|
```yaml
|
|
realtime_dashboard:
|
|
current_sprint:
|
|
- velocity_burndown: Actual vs planned
|
|
- quality_gates: Pass/fail status
|
|
- defect_trend: New vs resolved
|
|
- coverage_delta: Change from baseline
|
|
|
|
system_health:
|
|
- error_rate: Last 15 minutes
|
|
- response_time: P50, P95, P99
|
|
- availability: Current status
|
|
- active_incidents: Count and severity
|
|
```
|
|
|
|
### Historical Analytics
|
|
```python
|
|
historical_views = {
|
|
"quality_trends": {
|
|
"timeframes": ["daily", "weekly", "monthly", "quarterly"],
|
|
"metrics": ["coverage", "complexity", "defects", "velocity"],
|
|
"comparisons": ["period_over_period", "target_vs_actual"]
|
|
},
|
|
"pattern_analysis": {
|
|
"defect_patterns": "Common causes and times",
|
|
"performance_patterns": "Peak usage impacts",
|
|
"team_patterns": "Productivity cycles"
|
|
}
|
|
}
|
|
```
|
|
|
|
## Alert and Action Framework
|
|
|
|
### Alert Configuration
|
|
```python
|
|
alert_rules = {
|
|
"critical": {
|
|
"coverage_drop": "Coverage decreased >5%",
|
|
"build_failure": "3 consecutive failures",
|
|
"production_error": "Error rate >2%",
|
|
"sla_breach": "Response time >SLA"
|
|
},
|
|
"warning": {
|
|
"trend_negative": "3-day negative trend",
|
|
"threshold_approach": "Within 10% of limit",
|
|
"anomaly_detected": "Outside 2 std deviations"
|
|
}
|
|
}
|
|
|
|
def trigger_alert(metric, severity, value):
|
|
alert = {
|
|
"metric": metric,
|
|
"severity": severity,
|
|
"value": value,
|
|
"threshold": get_threshold(metric),
|
|
"action_required": get_required_action(metric, severity)
|
|
}
|
|
notify_stakeholders(alert)
|
|
```
|
|
|
|
### Action Tracking
|
|
```markdown
|
|
## Quality Action Item
|
|
**Metric**: {metric_name}
|
|
**Issue**: {threshold_violation}
|
|
**Severity**: {critical/high/medium}
|
|
**Detected**: {timestamp}
|
|
|
|
### Required Actions
|
|
1. **Immediate**: {emergency_action}
|
|
2. **Short-term**: {fix_action}
|
|
3. **Long-term**: {prevention_action}
|
|
|
|
### Tracking
|
|
- **Owner**: {responsible_person}
|
|
- **Due Date**: {deadline}
|
|
- **Status**: {in_progress/blocked/complete}
|
|
```
|
|
|
|
## Success Criteria
|
|
- 100% automated metric collection
|
|
- <5 minute data freshness
|
|
- Zero manual metric calculation
|
|
- 90% alert accuracy (not false positives)
|
|
- Action completion rate >95%
|
|
|
|
## Memory Integration
|
|
```python
|
|
# Quality metrics memory
|
|
quality_metrics_memory = {
|
|
"type": "quality_metrics_snapshot",
|
|
"timestamp": collection_time,
|
|
"metrics": {
|
|
"code_quality": code_metrics,
|
|
"process_quality": process_metrics,
|
|
"product_quality": product_metrics
|
|
},
|
|
"analysis": {
|
|
"trends": identified_trends,
|
|
"anomalies": detected_anomalies,
|
|
"correlations": metric_relationships,
|
|
"health_score": overall_score
|
|
},
|
|
"alerts": {
|
|
"triggered": alerts_sent,
|
|
"false_positives": incorrect_alerts,
|
|
"missed_issues": undetected_problems
|
|
},
|
|
"actions": {
|
|
"created": action_items_created,
|
|
"completed": actions_resolved,
|
|
"effectiveness": improvement_achieved
|
|
},
|
|
"insights": {
|
|
"patterns": recurring_patterns,
|
|
"predictions": forecast_accuracy,
|
|
"recommendations": suggested_improvements
|
|
}
|
|
}
|
|
```
|
|
|
|
## Metrics Report Template
|
|
```markdown
|
|
# Quality Metrics Report
|
|
**Period**: {start_date} - {end_date}
|
|
**Overall Health**: {score}/100
|
|
|
|
## Executive Summary
|
|
- **Quality Trend**: {improving/stable/declining}
|
|
- **Key Achievements**: {top_improvements}
|
|
- **Main Concerns**: {top_issues}
|
|
- **Action Items**: {count} ({completed}/{total})
|
|
|
|
## Detailed Metrics
|
|
|
|
### Code Quality
|
|
| Metric | Current | Target | Trend | Status |
|
|
|--------|---------|---------|--------|---------|
|
|
| Coverage | {n}% | {t}% | {↑↓→} | {🟢🟡🔴} |
|
|
| Complexity | {n} | {t} | {↑↓→} | {🟢🟡🔴} |
|
|
|
|
### Process Quality
|
|
| Metric | Current | Target | Trend | Status |
|
|
|--------|---------|---------|--------|---------|
|
|
| Velocity | {n} | {t} | {↑↓→} | {🟢🟡🔴} |
|
|
| Lead Time | {n}d | {t}d | {↑↓→} | {🟢🟡🔴} |
|
|
|
|
### Product Quality
|
|
| Metric | Current | Target | Trend | Status |
|
|
|--------|---------|---------|--------|---------|
|
|
| Availability | {n}% | {t}% | {↑↓→} | {🟢🟡🔴} |
|
|
| Error Rate | {n}% | {t}% | {↑↓→} | {🟢🟡🔴} |
|
|
|
|
## Insights & Patterns
|
|
1. **Finding**: {insight}
|
|
- Impact: {description}
|
|
- Recommendation: {action}
|
|
|
|
## Action Plan
|
|
| Action | Owner | Due Date | Status |
|
|
|--------|--------|----------|---------|
|
|
| {action} | {owner} | {date} | {status} |
|
|
|
|
## Next Period Focus
|
|
{key_areas_for_improvement}
|
|
```
|
|
|
|
## Brotherhood Collaboration
|
|
- Metric definition with all teams
|
|
- Threshold setting with stakeholders
|
|
- Alert configuration with ops team
|
|
- Action planning with leadership |