fix: resolve critical complexity routing issues from multi-agent review
Fixes 6 critical issues discovered in multi-agent code review: 1. **Parameter propagation** - CRITICAL FIX - Added complexity_level parameter to super-dev-pipeline invocations - Fixed both sequential and parallel execution paths - Without this, complexity routing was completely non-functional 2. **Keyword matching rules** - CRITICAL FIX - Defined explicit matching algorithm in workflow.yaml - Case insensitive, word boundary matching, exact strategy - Added keyword variants (auth -> authentication, authorize, etc.) - Scan locations: story_title, task_descriptions, subtask_descriptions 3. **Threshold decision tree** - CRITICAL FIX - Rewrote overlapping logic to be mutually exclusive - Priority order: COMPLEX → MICRO → STANDARD - Prevents stories from matching multiple categories 4. **Task counting method** - CRITICAL FIX - Defined method: "top_level_only" (ignore subtasks) - Added documentation with examples - Eliminates ambiguity in complexity scoring 5. **max_files implementation** - FIX - Added file_count ≤ 5 check to MICRO classification - Previously extracted but never used (dead code) 6. **Version synchronization** - FIX - Updated super-dev-pipeline to v1.3.0 (was 1.2.0) - Matches batch-super-dev version for consistency Impact: These fixes make complexity routing actually functional. The original implementation computed complexity but never passed it to the pipeline, rendering the entire feature non-operational.
This commit is contained in:
parent
e5ede9ec3f
commit
9bdf489438
|
|
@ -0,0 +1,184 @@
|
||||||
|
================================================================================
|
||||||
|
DOCUMENTATION ASSESSMENT SUMMARY - v1.3.0 Complexity Routing
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
Assessment Date: 2026-01-07
|
||||||
|
Commit: e5ede9ec (feat: add complexity-based routing and pipeline optimizations)
|
||||||
|
Files Analyzed: 9 changed files across batch-super-dev, super-dev-pipeline, dev-story
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
OVERALL FINDINGS
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
✅ STRENGTHS:
|
||||||
|
- Clear version markers (mostly consistent)
|
||||||
|
- Well-structured step-file architecture
|
||||||
|
- Consistent tone and professional style
|
||||||
|
- Good use of emoji for visual hierarchy
|
||||||
|
|
||||||
|
⚠️ ISSUES FOUND:
|
||||||
|
- 3 CRITICAL issues (blocking implementation)
|
||||||
|
- 2 HIGH priority issues (quality gates)
|
||||||
|
- 3 MEDIUM issues (consistency)
|
||||||
|
- Zero LOW priority issues requiring action
|
||||||
|
|
||||||
|
QUALITY SCORE: 7/10
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
CRITICAL ISSUES (Must Fix Before Merge)
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
1. TASK COUNTING METHOD UNDEFINED (Line 230)
|
||||||
|
File: batch-super-dev/instructions.md
|
||||||
|
Impact: Complexity scoring inconsistent across stories
|
||||||
|
Fix: Add explicit definition including nested subtasks handling
|
||||||
|
|
||||||
|
2. FILE_COUNT COLLECTED BUT UNUSED (Line 231)
|
||||||
|
File: batch-super-dev/instructions.md
|
||||||
|
Impact: Dead code, confusing developers
|
||||||
|
Fix: Either remove collection OR implement in complexity formula
|
||||||
|
|
||||||
|
3. OVERLAPPING COMPLEXITY THRESHOLDS (Lines 242-244)
|
||||||
|
File: batch-super-dev/instructions.md
|
||||||
|
Impact: Edge cases route incorrectly
|
||||||
|
Example: 4-task "security" story with HIGH keyword
|
||||||
|
Fix: Replace with explicit decision tree (see SUGGESTED_FIXES.md)
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
HIGH PRIORITY ISSUES (Quality & Integration)
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
4. MULTI-AGENT MERGE STRATEGY MISSING (Lines 38-74)
|
||||||
|
File: step-05-code-review.md
|
||||||
|
Impact: Different merge results depending on implementer
|
||||||
|
Fix: Document deduplication, precedence, and merge algorithm
|
||||||
|
|
||||||
|
5. RISK KEYWORD MATCHING UNDEFINED (Line 232)
|
||||||
|
File: batch-super-dev/instructions.md
|
||||||
|
Impact: ±5 points variance in complexity scoring
|
||||||
|
Issues: Case sensitivity, substring vs whole-word, deduplication
|
||||||
|
Fix: Specify exact matching rules with pseudocode
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
VERSION & CONSISTENCY ISSUES
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
6. VERSION MARKER MISMATCH (Line 94)
|
||||||
|
File: step-01-init.md
|
||||||
|
Current: "NEW v1.2.0"
|
||||||
|
Should: "NEW v1.3.0" (to match commit and batch-super-dev)
|
||||||
|
|
||||||
|
7. BAILOUT vs SKIP TERMINOLOGY (Sections 4.5 + 6)
|
||||||
|
File: step-01-init.md
|
||||||
|
Issue: Both use similar language but mean different things
|
||||||
|
Fix: Document distinction clearly (see SUGGESTED_FIXES.md)
|
||||||
|
|
||||||
|
8. CASE INCONSISTENCY (Multiple files)
|
||||||
|
Issue: "MICRO" vs "micro" used inconsistently
|
||||||
|
Fix: Document convention (uppercase display, lowercase storage)
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
DECISION TREE EXAMPLE (FIX #3)
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
Current (Problematic):
|
||||||
|
MICRO: task_count ≤ 3 AND complexity_score ≤ 5 AND no HIGH risk
|
||||||
|
COMPLEX: task_count ≥ 16 OR complexity_score ≥ 20 OR has HIGH risk
|
||||||
|
STANDARD: everything else
|
||||||
|
|
||||||
|
Proposed (Clear & Unambiguous):
|
||||||
|
Step 1: Has HIGH keyword? → COMPLEX (STOP)
|
||||||
|
Step 2: task_count ≥ 16 OR score ≥ 20? → COMPLEX (STOP)
|
||||||
|
Step 3: task_count ≤ 3 AND score ≤ 5? → MICRO (STOP)
|
||||||
|
Step 4: Default → STANDARD
|
||||||
|
|
||||||
|
Test Cases:
|
||||||
|
✓ 2-task UI story: MICRO
|
||||||
|
✓ 4-task auth story: COMPLEX (HIGH keyword)
|
||||||
|
✓ 8-task standard story: STANDARD
|
||||||
|
✓ 15-task database migration: COMPLEX (HIGH keywords)
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
DOCUMENTATION DELIVERABLES
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
1. DOCUMENTATION_ASSESSMENT.md (This analysis)
|
||||||
|
- Full assessment with detailed findings
|
||||||
|
- Quality ratings and metrics
|
||||||
|
- Test scenarios for complexity scoring
|
||||||
|
- 11 sections covering all aspects
|
||||||
|
|
||||||
|
2. CRITICAL_FIXES_REQUIRED.md
|
||||||
|
- Blocking issues only
|
||||||
|
- Business impact explanation
|
||||||
|
- Specific file paths and line numbers
|
||||||
|
- Sign-off checklist before merge
|
||||||
|
|
||||||
|
3. SUGGESTED_FIXES.md
|
||||||
|
- Exact replacement text for all 5 critical issues
|
||||||
|
- Code examples and pseudocode
|
||||||
|
- Complete working solutions ready to copy-paste
|
||||||
|
- 9 detailed fixes with implementation examples
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
RECOMMENDED ACTION PLAN
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
PHASE 1: Address Critical Issues (2-3 hours)
|
||||||
|
☐ Fix task counting method (FIX #1)
|
||||||
|
☐ Decide on file_count (FIX #2)
|
||||||
|
☐ Implement decision tree (FIX #3)
|
||||||
|
☐ Document merge strategy (FIX #4)
|
||||||
|
☐ Specify keyword matching (FIX #5)
|
||||||
|
|
||||||
|
PHASE 2: Fix Versions & Consistency (30 minutes)
|
||||||
|
☐ Update version markers (FIX #6)
|
||||||
|
☐ Document bailout/skip (FIX #7)
|
||||||
|
☐ Document case convention (FIX #8)
|
||||||
|
|
||||||
|
PHASE 3: QA & Testing (1-2 hours)
|
||||||
|
☐ Test complexity scoring edge cases
|
||||||
|
☐ Verify template rendering
|
||||||
|
☐ Review decision tree logic
|
||||||
|
☐ Check all file paths are absolute
|
||||||
|
|
||||||
|
PHASE 4: Review & Sign-Off (1 hour)
|
||||||
|
☐ PR review with focus on critical fixes
|
||||||
|
☐ Merge with confidence
|
||||||
|
|
||||||
|
Total Estimate: 4-6 hours
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
KEY METRICS
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
Clarity: 7/10 (Good, but some ambiguous sections)
|
||||||
|
Consistency: 7/10 (Mostly consistent, some case/version issues)
|
||||||
|
Completeness: 6/10 (Missing integration details)
|
||||||
|
Correctness: 6/10 (Critical logic issues in thresholds)
|
||||||
|
|
||||||
|
Files Affected: 4 core files
|
||||||
|
Lines Modified: ~150 lines across all fixes
|
||||||
|
Breaking Changes: None (backward compatible)
|
||||||
|
Risk Level: MEDIUM (issues are fixable, non-fundamental)
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
CONTACT & REFERENCES
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
Commit: e5ede9ec
|
||||||
|
Feature: Complexity-Based Routing v1.3.0
|
||||||
|
Assessment: 2026-01-07
|
||||||
|
|
||||||
|
Files to Review:
|
||||||
|
- src/modules/bmm/workflows/4-implementation/batch-super-dev/instructions.md
|
||||||
|
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-01-init.md
|
||||||
|
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-03-implement.md
|
||||||
|
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-04-post-validation.md
|
||||||
|
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-05-code-review.md
|
||||||
|
- src/modules/bmm/workflows/4-implementation/batch-super-dev/workflow.yaml
|
||||||
|
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
END OF SUMMARY
|
||||||
|
================================================================================
|
||||||
|
|
@ -227,21 +227,41 @@ Run `/bmad:bmm:workflows:sprint-status` to see status.
|
||||||
<substep n="2.6a" title="Analyze story complexity">
|
<substep n="2.6a" title="Analyze story complexity">
|
||||||
<action>Read story file: {{file_path}}</action>
|
<action>Read story file: {{file_path}}</action>
|
||||||
|
|
||||||
<action>Count unchecked tasks ([ ]) in Tasks/Subtasks section → task_count</action>
|
<action>Count unchecked tasks ([ ]) at top level only in Tasks/Subtasks section → task_count
|
||||||
|
(See workflow.yaml complexity.task_counting.method = "top_level_only")
|
||||||
|
</action>
|
||||||
<action>Extract file paths mentioned in tasks → file_count</action>
|
<action>Extract file paths mentioned in tasks → file_count</action>
|
||||||
<action>Scan story title and task descriptions for risk keywords</action>
|
<action>Scan story title and task descriptions for risk keywords using rules from workflow.yaml:
|
||||||
|
- Case insensitive matching (require_word_boundaries: true)
|
||||||
|
- Include keyword variants (e.g., "authentication" matches "auth")
|
||||||
|
- Scan: story_title, task_descriptions, subtask_descriptions
|
||||||
|
</action>
|
||||||
|
|
||||||
<action>Calculate complexity score:
|
<action>Calculate complexity score:
|
||||||
- Base score = task_count
|
- Base score = task_count
|
||||||
- Add 5 for each HIGH risk keyword (auth, security, payment, migration, database, schema)
|
- Add 5 for each HIGH risk keyword match (auth, security, payment, migration, database, schema, encryption)
|
||||||
- Add 2 for each MEDIUM risk keyword (api, integration, external, cache)
|
- Add 2 for each MEDIUM risk keyword match (api, integration, external, third-party, cache)
|
||||||
- Add 0 for LOW risk keywords (ui, style, config, docs, test)
|
- Add 0 for LOW risk keywords (ui, style, config, docs, test)
|
||||||
|
- Count each keyword only once (no duplicates)
|
||||||
</action>
|
</action>
|
||||||
|
|
||||||
<action>Assign complexity level:
|
<action>Assign complexity level using mutually exclusive decision tree (priority order):
|
||||||
- MICRO: task_count ≤ 3 AND complexity_score ≤ 5 AND no HIGH risk keywords
|
|
||||||
- COMPLEX: task_count ≥ 16 OR complexity_score ≥ 20 OR has HIGH risk keywords
|
1. Check COMPLEX first (highest priority):
|
||||||
- STANDARD: everything else
|
IF (task_count ≥ 16 OR complexity_score ≥ 20 OR has ANY HIGH risk keyword)
|
||||||
|
THEN level = COMPLEX
|
||||||
|
|
||||||
|
2. Else check MICRO (lowest complexity):
|
||||||
|
ELSE IF (task_count ≤ 3 AND complexity_score ≤ 5 AND file_count ≤ 5)
|
||||||
|
THEN level = MICRO
|
||||||
|
|
||||||
|
3. Else default to STANDARD:
|
||||||
|
ELSE level = STANDARD
|
||||||
|
|
||||||
|
This ensures no overlaps:
|
||||||
|
- Story with HIGH keyword → COMPLEX (never MICRO or STANDARD)
|
||||||
|
- Story with 4-15 tasks or >5 files → STANDARD (not MICRO or COMPLEX)
|
||||||
|
- Story with ≤3 tasks, ≤5 files, no HIGH keywords → MICRO
|
||||||
</action>
|
</action>
|
||||||
|
|
||||||
<action>Store complexity_level for story: {{story_key}}.complexity = {level, score, task_count, risk_keywords}</action>
|
<action>Store complexity_level for story: {{story_key}}.complexity = {level, score, task_count, risk_keywords}</action>
|
||||||
|
|
@ -418,7 +438,7 @@ Only the first {{max_stories}} will be processed.</output>
|
||||||
</output>
|
</output>
|
||||||
|
|
||||||
<action>Invoke workflow: /bmad:bmm:workflows:super-dev-pipeline</action>
|
<action>Invoke workflow: /bmad:bmm:workflows:super-dev-pipeline</action>
|
||||||
<action>Parameters: mode=batch, story_key={{story_key}}</action>
|
<action>Parameters: mode=batch, story_key={{story_key}}, complexity_level={{story_key}}.complexity.level</action>
|
||||||
|
|
||||||
<check if="super-dev-pipeline succeeded">
|
<check if="super-dev-pipeline succeeded">
|
||||||
<output>✅ Implementation complete: {{story_key}}</output>
|
<output>✅ Implementation complete: {{story_key}}</output>
|
||||||
|
|
@ -507,7 +527,7 @@ Spawning Task agents in parallel...
|
||||||
CRITICAL INSTRUCTIONS:
|
CRITICAL INSTRUCTIONS:
|
||||||
1. Load workflow.xml: _bmad/core/tasks/workflow.xml
|
1. Load workflow.xml: _bmad/core/tasks/workflow.xml
|
||||||
2. Load workflow config: _bmad/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml
|
2. Load workflow config: _bmad/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml
|
||||||
3. Execute in BATCH mode with story_key={{story_key}}
|
3. Execute in BATCH mode with story_key={{story_key}} and complexity_level={{story_key}}.complexity.level
|
||||||
4. Follow all 7 pipeline steps (init, pre-gap, implement, post-validate, code-review, complete, summary)
|
4. Follow all 7 pipeline steps (init, pre-gap, implement, post-validate, code-review, complete, summary)
|
||||||
5. Commit changes when complete
|
5. Commit changes when complete
|
||||||
6. Report final status (done/failed) with file list
|
6. Report final status (done/failed) with file list
|
||||||
|
|
|
||||||
|
|
@ -61,6 +61,33 @@ complexity:
|
||||||
medium: ["api", "integration", "external", "third-party", "cache"]
|
medium: ["api", "integration", "external", "third-party", "cache"]
|
||||||
low: ["ui", "style", "config", "docs", "test"]
|
low: ["ui", "style", "config", "docs", "test"]
|
||||||
|
|
||||||
|
# Keyword matching configuration (defines how risk keywords are detected)
|
||||||
|
keyword_matching:
|
||||||
|
case_sensitive: false # "AUTH" matches "auth"
|
||||||
|
require_word_boundaries: true # "auth" won't match "author"
|
||||||
|
match_strategy: "exact" # exact word match required (no stemming)
|
||||||
|
scan_locations:
|
||||||
|
- story_title
|
||||||
|
- task_descriptions
|
||||||
|
- subtask_descriptions
|
||||||
|
# Keyword variants (synonyms that map to canonical forms)
|
||||||
|
variants:
|
||||||
|
auth: ["authentication", "authorize", "authorization", "authz", "authn"]
|
||||||
|
database: ["db", "databases", "datastore"]
|
||||||
|
payment: ["payments", "pay", "billing", "checkout"]
|
||||||
|
migration: ["migrations", "migrate"]
|
||||||
|
security: ["secure", "security"]
|
||||||
|
encryption: ["encrypt", "encrypted", "cipher"]
|
||||||
|
|
||||||
|
# Task counting rules
|
||||||
|
task_counting:
|
||||||
|
method: "top_level_only" # Only count [ ] at task level, not subtasks
|
||||||
|
# Options: "top_level_only", "include_subtasks", "weighted"
|
||||||
|
# Example:
|
||||||
|
# - [ ] Parent task <- counts as 1
|
||||||
|
# - [ ] Subtask 1 <- ignored
|
||||||
|
# - [ ] Subtask 2 <- ignored
|
||||||
|
|
||||||
# Execution settings
|
# Execution settings
|
||||||
execution:
|
execution:
|
||||||
continue_on_failure: true # Keep processing remaining stories if one fails
|
continue_on_failure: true # Keep processing remaining stories if one fails
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
name: super-dev-pipeline
|
name: super-dev-pipeline
|
||||||
description: "Step-file architecture with complexity-based routing and smart batching. Micro stories get lightweight path, standard/complex get full quality gates."
|
description: "Step-file architecture with complexity-based routing and smart batching. Micro stories get lightweight path, standard/complex get full quality gates."
|
||||||
author: "BMad"
|
author: "BMad"
|
||||||
version: "1.2.0" # Added complexity-based routing
|
version: "1.3.0" # Synchronized with batch-super-dev for complexity-based routing
|
||||||
|
|
||||||
# Critical variables from config
|
# Critical variables from config
|
||||||
config_source: "{project-root}/_bmad/bmm/config.yaml"
|
config_source: "{project-root}/_bmad/bmm/config.yaml"
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue