refactor: consolidate 6 validation workflows into unified validate
Replaced: - validate-story - validate-story-deep - validate-all-stories - validate-all-stories-deep - validate-epic-status - validate-all-epics With single unified workflow: - validate (scope=story|epic|all, depth=quick|deep) 60KB of XML → ~8KB unified workflow.md
This commit is contained in:
Jonah Schulte
parent
cff4770c74
commit
323bee924e
|
|
@ -0,0 +1,353 @@
|
|||
# Validate v3.0 - Unified Story/Epic Validation
|
||||
|
||||
<purpose>
|
||||
Single workflow for all validation needs. Validates stories against codebase,
|
||||
detects false positives (checked but not implemented), and reports health scores.
|
||||
Read-only by default - does not modify files.
|
||||
</purpose>
|
||||
|
||||
<philosophy>
|
||||
**Trust But Verify**
|
||||
|
||||
- Quick mode: Checkbox counting, file existence, pattern matching
|
||||
- Deep mode: Haiku agents read actual code, verify implementation quality
|
||||
- Categorize: VERIFIED_COMPLETE, NEEDS_REWORK, FALSE_POSITIVE, IN_PROGRESS
|
||||
- Report accuracy gaps between claimed and actual completion
|
||||
</philosophy>
|
||||
|
||||
<config>
|
||||
name: validate
|
||||
version: 3.0.0
|
||||
|
||||
parameters:
|
||||
scope:
|
||||
story: "Single story file"
|
||||
epic: "All stories in an epic"
|
||||
all: "All stories in sprint"
|
||||
|
||||
target: "story_file path OR epic_number (depends on scope)"
|
||||
|
||||
depth:
|
||||
quick: "Checkbox counting, file existence checks"
|
||||
deep: "Haiku agents verify actual code implementation"
|
||||
|
||||
fix_mode: false # If true, update checkboxes and sprint-status
|
||||
|
||||
defaults:
|
||||
scope: story
|
||||
depth: quick
|
||||
fix_mode: false
|
||||
batch_size: 10
|
||||
model_for_deep: haiku
|
||||
|
||||
categories:
|
||||
VERIFIED_COMPLETE: {score: ">=95", false_positives: 0}
|
||||
COMPLETE_WITH_ISSUES: {score: ">=80", false_positives: "<=2"}
|
||||
FALSE_POSITIVE: {score: "<50", description: "Claimed done but missing code"}
|
||||
NEEDS_REWORK: {false_positives: ">2"}
|
||||
IN_PROGRESS: {description: "Accurately reflects partial completion"}
|
||||
NOT_STARTED: {checked: 0}
|
||||
</config>
|
||||
|
||||
<execution_context>
|
||||
@patterns/verification.md
|
||||
@patterns/hospital-grade.md
|
||||
</execution_context>
|
||||
|
||||
<process>
|
||||
|
||||
<step name="resolve_targets" priority="first">
|
||||
**Determine what to validate based on scope**
|
||||
|
||||
```
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
🔍 VALIDATION: {{scope}} scope, {{depth}} depth
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
```
|
||||
|
||||
**If scope == story:**
|
||||
```bash
|
||||
STORY_FILE="{{target}}"
|
||||
[ -f "$STORY_FILE" ] || { echo "❌ Story file not found"; exit 1; }
|
||||
```
|
||||
stories_to_validate = [target]
|
||||
|
||||
**If scope == epic:**
|
||||
```bash
|
||||
EPIC_NUM="{{target}}"
|
||||
# Get stories from sprint-status.yaml matching epic
|
||||
grep "^${EPIC_NUM}-" docs/sprint-artifacts/sprint-status.yaml
|
||||
```
|
||||
stories_to_validate = all stories starting with `{epic_num}-`
|
||||
|
||||
**If scope == all:**
|
||||
```bash
|
||||
# Find all story files, exclude meta-documents
|
||||
find docs/sprint-artifacts -name "*.md" | grep -v "EPIC-\|COMPLETION\|REPORT\|README"
|
||||
```
|
||||
stories_to_validate = all story files
|
||||
|
||||
Display:
|
||||
```
|
||||
Stories to validate: {{count}}
|
||||
Depth: {{depth}}
|
||||
{{#if depth == deep}}
|
||||
Estimated cost: ~${{count * 0.13}} (Haiku agents)
|
||||
{{/if}}
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="validate_quick" if="depth == quick">
|
||||
**Quick validation: checkbox counting and file checks**
|
||||
|
||||
For each story:
|
||||
|
||||
1. **Read story file** - Extract tasks, ACs, DoD
|
||||
2. **Count checkboxes:**
|
||||
```
|
||||
total_tasks = count of "- [ ]" and "- [x]"
|
||||
checked_tasks = count of "- [x]"
|
||||
completion_pct = checked / total × 100
|
||||
```
|
||||
|
||||
3. **Check file existence** (from Dev Agent Record):
|
||||
```bash
|
||||
for file in $FILE_LIST; do
|
||||
[ -f "$file" ] && echo "✅ $file" || echo "❌ $file MISSING"
|
||||
done
|
||||
```
|
||||
|
||||
4. **Basic stub detection:**
|
||||
```bash
|
||||
grep -l "TODO\|FIXME\|Not implemented\|throw new Error" $FILE_LIST
|
||||
```
|
||||
|
||||
5. **Categorize:**
|
||||
- ≥95% checked + files exist → VERIFIED_COMPLETE
|
||||
- ≥80% checked → COMPLETE_WITH_ISSUES
|
||||
- Files missing for checked tasks → FALSE_POSITIVE
|
||||
- <50% checked → IN_PROGRESS
|
||||
- 0% checked → NOT_STARTED
|
||||
|
||||
6. **Store result** with score and category
|
||||
</step>
|
||||
|
||||
<step name="validate_deep" if="depth == deep">
|
||||
**Deep validation: Haiku agents verify actual code**
|
||||
|
||||
For each story (or batch):
|
||||
|
||||
Spawn Haiku agent:
|
||||
```
|
||||
Task({
|
||||
subagent_type: "general-purpose",
|
||||
model: "haiku",
|
||||
description: "Deep validate {{story_id}}",
|
||||
prompt: `
|
||||
Verify ALL tasks for story {{story_id}} by reading actual code.
|
||||
|
||||
**Tasks to Verify:**
|
||||
{{#each tasks}}
|
||||
{{@index}}. [{{checked}}] {{text}}
|
||||
{{/each}}
|
||||
|
||||
**Files from Dev Agent Record:**
|
||||
{{file_list}}
|
||||
|
||||
**For EACH task:**
|
||||
1. Find relevant files (Glob)
|
||||
2. Read the files (Read tool)
|
||||
3. Verify: Is it real code or stubs? Tests exist? Error handling?
|
||||
4. Judge: actually_complete = true/false
|
||||
|
||||
**Return JSON:**
|
||||
{
|
||||
"story_id": "{{story_id}}",
|
||||
"tasks": [
|
||||
{
|
||||
"task_number": 0,
|
||||
"is_checked": true,
|
||||
"actually_complete": false,
|
||||
"confidence": "high",
|
||||
"evidence": "File has TODO on line 45",
|
||||
"issues": ["Stub implementation", "No tests"]
|
||||
}
|
||||
]
|
||||
}
|
||||
`
|
||||
})
|
||||
```
|
||||
|
||||
Parse results:
|
||||
- **False positive:** checked=true, actually_complete=false
|
||||
- **False negative:** checked=false, actually_complete=true
|
||||
- **Correct:** checked matches actually_complete
|
||||
|
||||
Calculate verification score:
|
||||
```
|
||||
score = (correct_count / total_count) × 100
|
||||
score -= (false_positive_count × 5) # Penalty for false positives
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="categorize_results">
|
||||
**Assign categories based on scores**
|
||||
|
||||
| Category | Criteria | Recommended Status |
|
||||
|----------|----------|-------------------|
|
||||
| VERIFIED_COMPLETE | score ≥95, FP=0 | done |
|
||||
| COMPLETE_WITH_ISSUES | score ≥80, FP≤2 | review |
|
||||
| FALSE_POSITIVE | score <50 OR FP>5 | in-progress |
|
||||
| NEEDS_REWORK | FP>2 | in-progress |
|
||||
| IN_PROGRESS | partial completion | in-progress |
|
||||
| NOT_STARTED | 0 checked | backlog |
|
||||
|
||||
For each story:
|
||||
- Compare current_status vs recommended_status
|
||||
- Flag if status is inaccurate
|
||||
</step>
|
||||
|
||||
<step name="aggregate_report" if="scope != story">
|
||||
**Generate batch summary**
|
||||
|
||||
```
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
📊 VALIDATION SUMMARY
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
Scope: {{scope}} {{#if target}}({{target}}){{/if}}
|
||||
Stories Validated: {{count}}
|
||||
Depth: {{depth}}
|
||||
|
||||
Overall Health Score: {{health_score}}/100
|
||||
|
||||
By Category:
|
||||
- ✅ VERIFIED_COMPLETE: {{verified_count}} ({{verified_pct}}%)
|
||||
- ⚠️ NEEDS_REWORK: {{rework_count}} ({{rework_pct}}%)
|
||||
- ❌ FALSE_POSITIVE: {{fp_count}} ({{fp_pct}}%)
|
||||
- 🔄 IN_PROGRESS: {{progress_count}} ({{progress_pct}}%)
|
||||
- 📋 NOT_STARTED: {{not_started_count}}
|
||||
|
||||
Task Accuracy:
|
||||
- Total tasks: {{total_tasks}}
|
||||
- False positives: {{fp_tasks}} (checked but not done)
|
||||
- False negatives: {{fn_tasks}} (done but not checked)
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
```
|
||||
|
||||
**If FALSE_POSITIVE stories found:**
|
||||
```
|
||||
❌ FALSE POSITIVE STORIES (Claimed Done, Not Implemented):
|
||||
|
||||
{{#each fp_stories}}
|
||||
- {{story_id}}: Score {{score}}/100, {{fp_task_count}} tasks missing
|
||||
Current: {{current_status}} → Should be: in-progress
|
||||
{{/each}}
|
||||
|
||||
Action: These stories need implementation work!
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="display_single_result" if="scope == story">
|
||||
**Show single story validation result**
|
||||
|
||||
```
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
📋 STORY VALIDATION: {{story_id}}
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
Epic: {{epic_num}}
|
||||
Current Status: {{current_status}}
|
||||
Recommended Status: {{recommended_status}}
|
||||
|
||||
Verification Score: {{score}}/100
|
||||
Category: {{category}}
|
||||
|
||||
Tasks: {{checked}}/{{total}} checked
|
||||
{{#if depth == deep}}
|
||||
- Verified complete: {{verified_count}}
|
||||
- False positives: {{fp_count}} (checked but code missing)
|
||||
- False negatives: {{fn_count}} (code exists, not checked)
|
||||
{{/if}}
|
||||
|
||||
{{#if category == "VERIFIED_COMPLETE"}}
|
||||
✅ Story is production-ready
|
||||
{{else if category == "FALSE_POSITIVE"}}
|
||||
❌ Story claimed done but has {{fp_count}} missing tasks
|
||||
Action: Update status to in-progress, implement missing code
|
||||
{{else if category == "NEEDS_REWORK"}}
|
||||
⚠️ Story needs rework: {{fp_count}} tasks with issues
|
||||
{{else if category == "IN_PROGRESS"}}
|
||||
🔄 Story accurately reflects partial completion
|
||||
{{/if}}
|
||||
|
||||
{{#if fp_tasks}}
|
||||
**False Positive Tasks (checked but not done):**
|
||||
{{#each fp_tasks}}
|
||||
- [ ] {{task}} — {{evidence}}
|
||||
{{/each}}
|
||||
{{/if}}
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="apply_fixes" if="fix_mode">
|
||||
**Auto-fix mode: update files based on validation**
|
||||
|
||||
```
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
🔧 AUTO-FIX MODE
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
```
|
||||
|
||||
**Fix false negatives** (code exists but unchecked):
|
||||
Use Edit tool to change `- [ ]` to `- [x]` for verified tasks
|
||||
|
||||
**Update sprint-status.yaml:**
|
||||
Use Edit tool to change status for inaccurate entries
|
||||
|
||||
**DO NOT auto-fix false positives** (requires implementation work)
|
||||
|
||||
```
|
||||
✅ Auto-fix complete:
|
||||
- {{fn_fixed}} false negatives checked
|
||||
- {{status_fixed}} statuses updated
|
||||
- {{fp_count}} false positives flagged (need manual implementation)
|
||||
```
|
||||
</step>
|
||||
|
||||
</process>
|
||||
|
||||
<examples>
|
||||
```bash
|
||||
# Validate single story (quick)
|
||||
/validate scope=story target=docs/sprint-artifacts/2-5-auth.md
|
||||
|
||||
# Validate single story (deep - uses Haiku)
|
||||
/validate scope=story target=docs/sprint-artifacts/2-5-auth.md depth=deep
|
||||
|
||||
# Validate all stories in epic 2
|
||||
/validate scope=epic target=2
|
||||
|
||||
# Validate all stories in epic 2 (deep)
|
||||
/validate scope=epic target=2 depth=deep
|
||||
|
||||
# Validate entire sprint
|
||||
/validate scope=all
|
||||
|
||||
# Validate and auto-fix false negatives
|
||||
/validate scope=all fix_mode=true
|
||||
```
|
||||
</examples>
|
||||
|
||||
<failure_handling>
|
||||
**Story file not found:** Skip with warning, continue batch.
|
||||
**Haiku agent fails:** Fall back to quick validation for that story.
|
||||
**All stories fail:** Report systemic issue, halt.
|
||||
</failure_handling>
|
||||
|
||||
<success_criteria>
|
||||
- [ ] All target stories validated
|
||||
- [ ] Categories assigned based on scores
|
||||
- [ ] False positives identified
|
||||
- [ ] Report generated
|
||||
- [ ] Fixes applied (if fix_mode=true)
|
||||
</success_criteria>
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
name: validate
|
||||
description: "Unified validation workflow. Validates stories against codebase, detects false positives, reports health scores. Replaces validate-story, validate-story-deep, validate-all-stories, validate-all-stories-deep, validate-epic-status, validate-all-epics."
|
||||
author: "BMad"
|
||||
version: "3.0.0"
|
||||
|
||||
# Critical variables from config
|
||||
config_source: "{project-root}/_bmad/bmm/config.yaml"
|
||||
implementation_artifacts: "{config_source}:implementation_artifacts"
|
||||
story_dir: "{implementation_artifacts}"
|
||||
sprint_status: "{implementation_artifacts}/sprint-status.yaml"
|
||||
|
||||
# Workflow components
|
||||
installed_path: "{project-root}/_bmad/bmm/workflows/4-implementation/validate"
|
||||
instructions: "{installed_path}/workflow.md"
|
||||
|
||||
# Input variables
|
||||
variables:
|
||||
scope: "story" # story | epic | all
|
||||
target: "" # story_file path OR epic_number (depends on scope)
|
||||
depth: "quick" # quick | deep (deep uses Haiku agents)
|
||||
fix_mode: false # If true, auto-fix false negatives and update statuses
|
||||
|
||||
# Deep validation settings
|
||||
deep_validation:
|
||||
model: "haiku"
|
||||
batch_size: 10
|
||||
cost_per_story: 0.13
|
||||
|
||||
# Output
|
||||
default_output_file: "{story_dir}/.validation-{scope}-{date}.md"
|
||||
|
||||
standalone: true
|
||||
web_bundle: false
|
||||
|
|
@ -0,0 +1,353 @@
|
|||
# Validate v3.0 - Unified Story/Epic Validation
|
||||
|
||||
<purpose>
|
||||
Single workflow for all validation needs. Validates stories against codebase,
|
||||
detects false positives (checked but not implemented), and reports health scores.
|
||||
Read-only by default - does not modify files.
|
||||
</purpose>
|
||||
|
||||
<philosophy>
|
||||
**Trust But Verify**
|
||||
|
||||
- Quick mode: Checkbox counting, file existence, pattern matching
|
||||
- Deep mode: Haiku agents read actual code, verify implementation quality
|
||||
- Categorize: VERIFIED_COMPLETE, NEEDS_REWORK, FALSE_POSITIVE, IN_PROGRESS
|
||||
- Report accuracy gaps between claimed and actual completion
|
||||
</philosophy>
|
||||
|
||||
<config>
|
||||
name: validate
|
||||
version: 3.0.0
|
||||
|
||||
parameters:
|
||||
scope:
|
||||
story: "Single story file"
|
||||
epic: "All stories in an epic"
|
||||
all: "All stories in sprint"
|
||||
|
||||
target: "story_file path OR epic_number (depends on scope)"
|
||||
|
||||
depth:
|
||||
quick: "Checkbox counting, file existence checks"
|
||||
deep: "Haiku agents verify actual code implementation"
|
||||
|
||||
fix_mode: false # If true, update checkboxes and sprint-status
|
||||
|
||||
defaults:
|
||||
scope: story
|
||||
depth: quick
|
||||
fix_mode: false
|
||||
batch_size: 10
|
||||
model_for_deep: haiku
|
||||
|
||||
categories:
|
||||
VERIFIED_COMPLETE: {score: ">=95", false_positives: 0}
|
||||
COMPLETE_WITH_ISSUES: {score: ">=80", false_positives: "<=2"}
|
||||
FALSE_POSITIVE: {score: "<50", description: "Claimed done but missing code"}
|
||||
NEEDS_REWORK: {false_positives: ">2"}
|
||||
IN_PROGRESS: {description: "Accurately reflects partial completion"}
|
||||
NOT_STARTED: {checked: 0}
|
||||
</config>
|
||||
|
||||
<execution_context>
|
||||
@patterns/verification.md
|
||||
@patterns/hospital-grade.md
|
||||
</execution_context>
|
||||
|
||||
<process>
|
||||
|
||||
<step name="resolve_targets" priority="first">
|
||||
**Determine what to validate based on scope**
|
||||
|
||||
```
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
🔍 VALIDATION: {{scope}} scope, {{depth}} depth
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
```
|
||||
|
||||
**If scope == story:**
|
||||
```bash
|
||||
STORY_FILE="{{target}}"
|
||||
[ -f "$STORY_FILE" ] || { echo "❌ Story file not found"; exit 1; }
|
||||
```
|
||||
stories_to_validate = [target]
|
||||
|
||||
**If scope == epic:**
|
||||
```bash
|
||||
EPIC_NUM="{{target}}"
|
||||
# Get stories from sprint-status.yaml matching epic
|
||||
grep "^${EPIC_NUM}-" docs/sprint-artifacts/sprint-status.yaml
|
||||
```
|
||||
stories_to_validate = all stories starting with `{epic_num}-`
|
||||
|
||||
**If scope == all:**
|
||||
```bash
|
||||
# Find all story files, exclude meta-documents
|
||||
find docs/sprint-artifacts -name "*.md" | grep -v "EPIC-\|COMPLETION\|REPORT\|README"
|
||||
```
|
||||
stories_to_validate = all story files
|
||||
|
||||
Display:
|
||||
```
|
||||
Stories to validate: {{count}}
|
||||
Depth: {{depth}}
|
||||
{{#if depth == deep}}
|
||||
Estimated cost: ~${{count * 0.13}} (Haiku agents)
|
||||
{{/if}}
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="validate_quick" if="depth == quick">
|
||||
**Quick validation: checkbox counting and file checks**
|
||||
|
||||
For each story:
|
||||
|
||||
1. **Read story file** - Extract tasks, ACs, DoD
|
||||
2. **Count checkboxes:**
|
||||
```
|
||||
total_tasks = count of "- [ ]" and "- [x]"
|
||||
checked_tasks = count of "- [x]"
|
||||
completion_pct = checked / total × 100
|
||||
```
|
||||
|
||||
3. **Check file existence** (from Dev Agent Record):
|
||||
```bash
|
||||
for file in $FILE_LIST; do
|
||||
[ -f "$file" ] && echo "✅ $file" || echo "❌ $file MISSING"
|
||||
done
|
||||
```
|
||||
|
||||
4. **Basic stub detection:**
|
||||
```bash
|
||||
grep -l "TODO\|FIXME\|Not implemented\|throw new Error" $FILE_LIST
|
||||
```
|
||||
|
||||
5. **Categorize:**
|
||||
- ≥95% checked + files exist → VERIFIED_COMPLETE
|
||||
- ≥80% checked → COMPLETE_WITH_ISSUES
|
||||
- Files missing for checked tasks → FALSE_POSITIVE
|
||||
- <50% checked → IN_PROGRESS
|
||||
- 0% checked → NOT_STARTED
|
||||
|
||||
6. **Store result** with score and category
|
||||
</step>
|
||||
|
||||
<step name="validate_deep" if="depth == deep">
|
||||
**Deep validation: Haiku agents verify actual code**
|
||||
|
||||
For each story (or batch):
|
||||
|
||||
Spawn Haiku agent:
|
||||
```
|
||||
Task({
|
||||
subagent_type: "general-purpose",
|
||||
model: "haiku",
|
||||
description: "Deep validate {{story_id}}",
|
||||
prompt: `
|
||||
Verify ALL tasks for story {{story_id}} by reading actual code.
|
||||
|
||||
**Tasks to Verify:**
|
||||
{{#each tasks}}
|
||||
{{@index}}. [{{checked}}] {{text}}
|
||||
{{/each}}
|
||||
|
||||
**Files from Dev Agent Record:**
|
||||
{{file_list}}
|
||||
|
||||
**For EACH task:**
|
||||
1. Find relevant files (Glob)
|
||||
2. Read the files (Read tool)
|
||||
3. Verify: Is it real code or stubs? Tests exist? Error handling?
|
||||
4. Judge: actually_complete = true/false
|
||||
|
||||
**Return JSON:**
|
||||
{
|
||||
"story_id": "{{story_id}}",
|
||||
"tasks": [
|
||||
{
|
||||
"task_number": 0,
|
||||
"is_checked": true,
|
||||
"actually_complete": false,
|
||||
"confidence": "high",
|
||||
"evidence": "File has TODO on line 45",
|
||||
"issues": ["Stub implementation", "No tests"]
|
||||
}
|
||||
]
|
||||
}
|
||||
`
|
||||
})
|
||||
```
|
||||
|
||||
Parse results:
|
||||
- **False positive:** checked=true, actually_complete=false
|
||||
- **False negative:** checked=false, actually_complete=true
|
||||
- **Correct:** checked matches actually_complete
|
||||
|
||||
Calculate verification score:
|
||||
```
|
||||
score = (correct_count / total_count) × 100
|
||||
score -= (false_positive_count × 5) # Penalty for false positives
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="categorize_results">
|
||||
**Assign categories based on scores**
|
||||
|
||||
| Category | Criteria | Recommended Status |
|
||||
|----------|----------|-------------------|
|
||||
| VERIFIED_COMPLETE | score ≥95, FP=0 | done |
|
||||
| COMPLETE_WITH_ISSUES | score ≥80, FP≤2 | review |
|
||||
| FALSE_POSITIVE | score <50 OR FP>5 | in-progress |
|
||||
| NEEDS_REWORK | FP>2 | in-progress |
|
||||
| IN_PROGRESS | partial completion | in-progress |
|
||||
| NOT_STARTED | 0 checked | backlog |
|
||||
|
||||
For each story:
|
||||
- Compare current_status vs recommended_status
|
||||
- Flag if status is inaccurate
|
||||
</step>
|
||||
|
||||
<step name="aggregate_report" if="scope != story">
|
||||
**Generate batch summary**
|
||||
|
||||
```
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
📊 VALIDATION SUMMARY
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
Scope: {{scope}} {{#if target}}({{target}}){{/if}}
|
||||
Stories Validated: {{count}}
|
||||
Depth: {{depth}}
|
||||
|
||||
Overall Health Score: {{health_score}}/100
|
||||
|
||||
By Category:
|
||||
- ✅ VERIFIED_COMPLETE: {{verified_count}} ({{verified_pct}}%)
|
||||
- ⚠️ NEEDS_REWORK: {{rework_count}} ({{rework_pct}}%)
|
||||
- ❌ FALSE_POSITIVE: {{fp_count}} ({{fp_pct}}%)
|
||||
- 🔄 IN_PROGRESS: {{progress_count}} ({{progress_pct}}%)
|
||||
- 📋 NOT_STARTED: {{not_started_count}}
|
||||
|
||||
Task Accuracy:
|
||||
- Total tasks: {{total_tasks}}
|
||||
- False positives: {{fp_tasks}} (checked but not done)
|
||||
- False negatives: {{fn_tasks}} (done but not checked)
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
```
|
||||
|
||||
**If FALSE_POSITIVE stories found:**
|
||||
```
|
||||
❌ FALSE POSITIVE STORIES (Claimed Done, Not Implemented):
|
||||
|
||||
{{#each fp_stories}}
|
||||
- {{story_id}}: Score {{score}}/100, {{fp_task_count}} tasks missing
|
||||
Current: {{current_status}} → Should be: in-progress
|
||||
{{/each}}
|
||||
|
||||
Action: These stories need implementation work!
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="display_single_result" if="scope == story">
|
||||
**Show single story validation result**
|
||||
|
||||
```
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
📋 STORY VALIDATION: {{story_id}}
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
Epic: {{epic_num}}
|
||||
Current Status: {{current_status}}
|
||||
Recommended Status: {{recommended_status}}
|
||||
|
||||
Verification Score: {{score}}/100
|
||||
Category: {{category}}
|
||||
|
||||
Tasks: {{checked}}/{{total}} checked
|
||||
{{#if depth == deep}}
|
||||
- Verified complete: {{verified_count}}
|
||||
- False positives: {{fp_count}} (checked but code missing)
|
||||
- False negatives: {{fn_count}} (code exists, not checked)
|
||||
{{/if}}
|
||||
|
||||
{{#if category == "VERIFIED_COMPLETE"}}
|
||||
✅ Story is production-ready
|
||||
{{else if category == "FALSE_POSITIVE"}}
|
||||
❌ Story claimed done but has {{fp_count}} missing tasks
|
||||
Action: Update status to in-progress, implement missing code
|
||||
{{else if category == "NEEDS_REWORK"}}
|
||||
⚠️ Story needs rework: {{fp_count}} tasks with issues
|
||||
{{else if category == "IN_PROGRESS"}}
|
||||
🔄 Story accurately reflects partial completion
|
||||
{{/if}}
|
||||
|
||||
{{#if fp_tasks}}
|
||||
**False Positive Tasks (checked but not done):**
|
||||
{{#each fp_tasks}}
|
||||
- [ ] {{task}} — {{evidence}}
|
||||
{{/each}}
|
||||
{{/if}}
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="apply_fixes" if="fix_mode">
|
||||
**Auto-fix mode: update files based on validation**
|
||||
|
||||
```
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
🔧 AUTO-FIX MODE
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
```
|
||||
|
||||
**Fix false negatives** (code exists but unchecked):
|
||||
Use Edit tool to change `- [ ]` to `- [x]` for verified tasks
|
||||
|
||||
**Update sprint-status.yaml:**
|
||||
Use Edit tool to change status for inaccurate entries
|
||||
|
||||
**DO NOT auto-fix false positives** (requires implementation work)
|
||||
|
||||
```
|
||||
✅ Auto-fix complete:
|
||||
- {{fn_fixed}} false negatives checked
|
||||
- {{status_fixed}} statuses updated
|
||||
- {{fp_count}} false positives flagged (need manual implementation)
|
||||
```
|
||||
</step>
|
||||
|
||||
</process>
|
||||
|
||||
<examples>
|
||||
```bash
|
||||
# Validate single story (quick)
|
||||
/validate scope=story target=docs/sprint-artifacts/2-5-auth.md
|
||||
|
||||
# Validate single story (deep - uses Haiku)
|
||||
/validate scope=story target=docs/sprint-artifacts/2-5-auth.md depth=deep
|
||||
|
||||
# Validate all stories in epic 2
|
||||
/validate scope=epic target=2
|
||||
|
||||
# Validate all stories in epic 2 (deep)
|
||||
/validate scope=epic target=2 depth=deep
|
||||
|
||||
# Validate entire sprint
|
||||
/validate scope=all
|
||||
|
||||
# Validate and auto-fix false negatives
|
||||
/validate scope=all fix_mode=true
|
||||
```
|
||||
</examples>
|
||||
|
||||
<failure_handling>
|
||||
**Story file not found:** Skip with warning, continue batch.
|
||||
**Haiku agent fails:** Fall back to quick validation for that story.
|
||||
**All stories fail:** Report systemic issue, halt.
|
||||
</failure_handling>
|
||||
|
||||
<success_criteria>
|
||||
- [ ] All target stories validated
|
||||
- [ ] Categories assigned based on scores
|
||||
- [ ] False positives identified
|
||||
- [ ] Report generated
|
||||
- [ ] Fixes applied (if fix_mode=true)
|
||||
</success_criteria>
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
name: validate
|
||||
description: "Unified validation workflow. Validates stories against codebase, detects false positives, reports health scores. Replaces validate-story, validate-story-deep, validate-all-stories, validate-all-stories-deep, validate-epic-status, validate-all-epics."
|
||||
author: "BMad"
|
||||
version: "3.0.0"
|
||||
|
||||
# Critical variables from config
|
||||
config_source: "{project-root}/_bmad/bmm/config.yaml"
|
||||
implementation_artifacts: "{config_source}:implementation_artifacts"
|
||||
story_dir: "{implementation_artifacts}"
|
||||
sprint_status: "{implementation_artifacts}/sprint-status.yaml"
|
||||
|
||||
# Workflow components
|
||||
installed_path: "{project-root}/_bmad/bmm/workflows/4-implementation/validate"
|
||||
instructions: "{installed_path}/workflow.md"
|
||||
|
||||
# Input variables
|
||||
variables:
|
||||
scope: "story" # story | epic | all
|
||||
target: "" # story_file path OR epic_number (depends on scope)
|
||||
depth: "quick" # quick | deep (deep uses Haiku agents)
|
||||
fix_mode: false # If true, auto-fix false negatives and update statuses
|
||||
|
||||
# Deep validation settings
|
||||
deep_validation:
|
||||
model: "haiku"
|
||||
batch_size: 10
|
||||
cost_per_story: 0.13
|
||||
|
||||
# Output
|
||||
default_output_file: "{story_dir}/.validation-{scope}-{date}.md"
|
||||
|
||||
standalone: true
|
||||
web_bundle: false
|
||||
Loading…
Reference in New Issue