fix: resolve critical complexity routing issues from multi-agent review
Fixes 6 critical issues discovered in multi-agent code review: 1. **Parameter propagation** - CRITICAL FIX - Added complexity_level parameter to super-dev-pipeline invocations - Fixed both sequential and parallel execution paths - Without this, complexity routing was completely non-functional 2. **Keyword matching rules** - CRITICAL FIX - Defined explicit matching algorithm in workflow.yaml - Case insensitive, word boundary matching, exact strategy - Added keyword variants (auth -> authentication, authorize, etc.) - Scan locations: story_title, task_descriptions, subtask_descriptions 3. **Threshold decision tree** - CRITICAL FIX - Rewrote overlapping logic to be mutually exclusive - Priority order: COMPLEX → MICRO → STANDARD - Prevents stories from matching multiple categories 4. **Task counting method** - CRITICAL FIX - Defined method: "top_level_only" (ignore subtasks) - Added documentation with examples - Eliminates ambiguity in complexity scoring 5. **max_files implementation** - FIX - Added file_count ≤ 5 check to MICRO classification - Previously extracted but never used (dead code) 6. **Version synchronization** - FIX - Updated super-dev-pipeline to v1.3.0 (was 1.2.0) - Matches batch-super-dev version for consistency Impact: These fixes make complexity routing actually functional. The original implementation computed complexity but never passed it to the pipeline, rendering the entire feature non-operational.
This commit is contained in:
Jonah Schulte
parent
e5ede9ec3f
commit
9bdf489438
|
|
@ -0,0 +1,184 @@
|
|||
================================================================================
|
||||
DOCUMENTATION ASSESSMENT SUMMARY - v1.3.0 Complexity Routing
|
||||
================================================================================
|
||||
|
||||
Assessment Date: 2026-01-07
|
||||
Commit: e5ede9ec (feat: add complexity-based routing and pipeline optimizations)
|
||||
Files Analyzed: 9 changed files across batch-super-dev, super-dev-pipeline, dev-story
|
||||
|
||||
================================================================================
|
||||
OVERALL FINDINGS
|
||||
================================================================================
|
||||
|
||||
✅ STRENGTHS:
|
||||
- Clear version markers (mostly consistent)
|
||||
- Well-structured step-file architecture
|
||||
- Consistent tone and professional style
|
||||
- Good use of emoji for visual hierarchy
|
||||
|
||||
⚠️ ISSUES FOUND:
|
||||
- 3 CRITICAL issues (blocking implementation)
|
||||
- 2 HIGH priority issues (quality gates)
|
||||
- 3 MEDIUM issues (consistency)
|
||||
- Zero LOW priority issues requiring action
|
||||
|
||||
QUALITY SCORE: 7/10
|
||||
|
||||
================================================================================
|
||||
CRITICAL ISSUES (Must Fix Before Merge)
|
||||
================================================================================
|
||||
|
||||
1. TASK COUNTING METHOD UNDEFINED (Line 230)
|
||||
File: batch-super-dev/instructions.md
|
||||
Impact: Complexity scoring inconsistent across stories
|
||||
Fix: Add explicit definition including nested subtasks handling
|
||||
|
||||
2. FILE_COUNT COLLECTED BUT UNUSED (Line 231)
|
||||
File: batch-super-dev/instructions.md
|
||||
Impact: Dead code, confusing developers
|
||||
Fix: Either remove collection OR implement in complexity formula
|
||||
|
||||
3. OVERLAPPING COMPLEXITY THRESHOLDS (Lines 242-244)
|
||||
File: batch-super-dev/instructions.md
|
||||
Impact: Edge cases route incorrectly
|
||||
Example: 4-task "security" story with HIGH keyword
|
||||
Fix: Replace with explicit decision tree (see SUGGESTED_FIXES.md)
|
||||
|
||||
================================================================================
|
||||
HIGH PRIORITY ISSUES (Quality & Integration)
|
||||
================================================================================
|
||||
|
||||
4. MULTI-AGENT MERGE STRATEGY MISSING (Lines 38-74)
|
||||
File: step-05-code-review.md
|
||||
Impact: Different merge results depending on implementer
|
||||
Fix: Document deduplication, precedence, and merge algorithm
|
||||
|
||||
5. RISK KEYWORD MATCHING UNDEFINED (Line 232)
|
||||
File: batch-super-dev/instructions.md
|
||||
Impact: ±5 points variance in complexity scoring
|
||||
Issues: Case sensitivity, substring vs whole-word, deduplication
|
||||
Fix: Specify exact matching rules with pseudocode
|
||||
|
||||
================================================================================
|
||||
VERSION & CONSISTENCY ISSUES
|
||||
================================================================================
|
||||
|
||||
6. VERSION MARKER MISMATCH (Line 94)
|
||||
File: step-01-init.md
|
||||
Current: "NEW v1.2.0"
|
||||
Should: "NEW v1.3.0" (to match commit and batch-super-dev)
|
||||
|
||||
7. BAILOUT vs SKIP TERMINOLOGY (Sections 4.5 + 6)
|
||||
File: step-01-init.md
|
||||
Issue: Both use similar language but mean different things
|
||||
Fix: Document distinction clearly (see SUGGESTED_FIXES.md)
|
||||
|
||||
8. CASE INCONSISTENCY (Multiple files)
|
||||
Issue: "MICRO" vs "micro" used inconsistently
|
||||
Fix: Document convention (uppercase display, lowercase storage)
|
||||
|
||||
================================================================================
|
||||
DECISION TREE EXAMPLE (FIX #3)
|
||||
================================================================================
|
||||
|
||||
Current (Problematic):
|
||||
MICRO: task_count ≤ 3 AND complexity_score ≤ 5 AND no HIGH risk
|
||||
COMPLEX: task_count ≥ 16 OR complexity_score ≥ 20 OR has HIGH risk
|
||||
STANDARD: everything else
|
||||
|
||||
Proposed (Clear & Unambiguous):
|
||||
Step 1: Has HIGH keyword? → COMPLEX (STOP)
|
||||
Step 2: task_count ≥ 16 OR score ≥ 20? → COMPLEX (STOP)
|
||||
Step 3: task_count ≤ 3 AND score ≤ 5? → MICRO (STOP)
|
||||
Step 4: Default → STANDARD
|
||||
|
||||
Test Cases:
|
||||
✓ 2-task UI story: MICRO
|
||||
✓ 4-task auth story: COMPLEX (HIGH keyword)
|
||||
✓ 8-task standard story: STANDARD
|
||||
✓ 15-task database migration: COMPLEX (HIGH keywords)
|
||||
|
||||
================================================================================
|
||||
DOCUMENTATION DELIVERABLES
|
||||
================================================================================
|
||||
|
||||
1. DOCUMENTATION_ASSESSMENT.md (This analysis)
|
||||
- Full assessment with detailed findings
|
||||
- Quality ratings and metrics
|
||||
- Test scenarios for complexity scoring
|
||||
- 11 sections covering all aspects
|
||||
|
||||
2. CRITICAL_FIXES_REQUIRED.md
|
||||
- Blocking issues only
|
||||
- Business impact explanation
|
||||
- Specific file paths and line numbers
|
||||
- Sign-off checklist before merge
|
||||
|
||||
3. SUGGESTED_FIXES.md
|
||||
- Exact replacement text for all 5 critical issues
|
||||
- Code examples and pseudocode
|
||||
- Complete working solutions ready to copy-paste
|
||||
- 9 detailed fixes with implementation examples
|
||||
|
||||
================================================================================
|
||||
RECOMMENDED ACTION PLAN
|
||||
================================================================================
|
||||
|
||||
PHASE 1: Address Critical Issues (2-3 hours)
|
||||
☐ Fix task counting method (FIX #1)
|
||||
☐ Decide on file_count (FIX #2)
|
||||
☐ Implement decision tree (FIX #3)
|
||||
☐ Document merge strategy (FIX #4)
|
||||
☐ Specify keyword matching (FIX #5)
|
||||
|
||||
PHASE 2: Fix Versions & Consistency (30 minutes)
|
||||
☐ Update version markers (FIX #6)
|
||||
☐ Document bailout/skip (FIX #7)
|
||||
☐ Document case convention (FIX #8)
|
||||
|
||||
PHASE 3: QA & Testing (1-2 hours)
|
||||
☐ Test complexity scoring edge cases
|
||||
☐ Verify template rendering
|
||||
☐ Review decision tree logic
|
||||
☐ Check all file paths are absolute
|
||||
|
||||
PHASE 4: Review & Sign-Off (1 hour)
|
||||
☐ PR review with focus on critical fixes
|
||||
☐ Merge with confidence
|
||||
|
||||
Total Estimate: 4-6 hours
|
||||
|
||||
================================================================================
|
||||
KEY METRICS
|
||||
================================================================================
|
||||
|
||||
Clarity: 7/10 (Good, but some ambiguous sections)
|
||||
Consistency: 7/10 (Mostly consistent, some case/version issues)
|
||||
Completeness: 6/10 (Missing integration details)
|
||||
Correctness: 6/10 (Critical logic issues in thresholds)
|
||||
|
||||
Files Affected: 4 core files
|
||||
Lines Modified: ~150 lines across all fixes
|
||||
Breaking Changes: None (backward compatible)
|
||||
Risk Level: MEDIUM (issues are fixable, non-fundamental)
|
||||
|
||||
================================================================================
|
||||
CONTACT & REFERENCES
|
||||
================================================================================
|
||||
|
||||
Commit: e5ede9ec
|
||||
Feature: Complexity-Based Routing v1.3.0
|
||||
Assessment: 2026-01-07
|
||||
|
||||
Files to Review:
|
||||
- src/modules/bmm/workflows/4-implementation/batch-super-dev/instructions.md
|
||||
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-01-init.md
|
||||
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-03-implement.md
|
||||
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-04-post-validation.md
|
||||
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-05-code-review.md
|
||||
- src/modules/bmm/workflows/4-implementation/batch-super-dev/workflow.yaml
|
||||
- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml
|
||||
|
||||
================================================================================
|
||||
END OF SUMMARY
|
||||
================================================================================
|
||||
|
|
@ -227,21 +227,41 @@ Run `/bmad:bmm:workflows:sprint-status` to see status.
|
|||
<substep n="2.6a" title="Analyze story complexity">
|
||||
<action>Read story file: {{file_path}}</action>
|
||||
|
||||
<action>Count unchecked tasks ([ ]) in Tasks/Subtasks section → task_count</action>
|
||||
<action>Count unchecked tasks ([ ]) at top level only in Tasks/Subtasks section → task_count
|
||||
(See workflow.yaml complexity.task_counting.method = "top_level_only")
|
||||
</action>
|
||||
<action>Extract file paths mentioned in tasks → file_count</action>
|
||||
<action>Scan story title and task descriptions for risk keywords</action>
|
||||
<action>Scan story title and task descriptions for risk keywords using rules from workflow.yaml:
|
||||
- Case insensitive matching (require_word_boundaries: true)
|
||||
- Include keyword variants (e.g., "authentication" matches "auth")
|
||||
- Scan: story_title, task_descriptions, subtask_descriptions
|
||||
</action>
|
||||
|
||||
<action>Calculate complexity score:
|
||||
- Base score = task_count
|
||||
- Add 5 for each HIGH risk keyword (auth, security, payment, migration, database, schema)
|
||||
- Add 2 for each MEDIUM risk keyword (api, integration, external, cache)
|
||||
- Add 5 for each HIGH risk keyword match (auth, security, payment, migration, database, schema, encryption)
|
||||
- Add 2 for each MEDIUM risk keyword match (api, integration, external, third-party, cache)
|
||||
- Add 0 for LOW risk keywords (ui, style, config, docs, test)
|
||||
- Count each keyword only once (no duplicates)
|
||||
</action>
|
||||
|
||||
<action>Assign complexity level:
|
||||
- MICRO: task_count ≤ 3 AND complexity_score ≤ 5 AND no HIGH risk keywords
|
||||
- COMPLEX: task_count ≥ 16 OR complexity_score ≥ 20 OR has HIGH risk keywords
|
||||
- STANDARD: everything else
|
||||
<action>Assign complexity level using mutually exclusive decision tree (priority order):
|
||||
|
||||
1. Check COMPLEX first (highest priority):
|
||||
IF (task_count ≥ 16 OR complexity_score ≥ 20 OR has ANY HIGH risk keyword)
|
||||
THEN level = COMPLEX
|
||||
|
||||
2. Else check MICRO (lowest complexity):
|
||||
ELSE IF (task_count ≤ 3 AND complexity_score ≤ 5 AND file_count ≤ 5)
|
||||
THEN level = MICRO
|
||||
|
||||
3. Else default to STANDARD:
|
||||
ELSE level = STANDARD
|
||||
|
||||
This ensures no overlaps:
|
||||
- Story with HIGH keyword → COMPLEX (never MICRO or STANDARD)
|
||||
- Story with 4-15 tasks or >5 files → STANDARD (not MICRO or COMPLEX)
|
||||
- Story with ≤3 tasks, ≤5 files, no HIGH keywords → MICRO
|
||||
</action>
|
||||
|
||||
<action>Store complexity_level for story: {{story_key}}.complexity = {level, score, task_count, risk_keywords}</action>
|
||||
|
|
@ -418,7 +438,7 @@ Only the first {{max_stories}} will be processed.</output>
|
|||
</output>
|
||||
|
||||
<action>Invoke workflow: /bmad:bmm:workflows:super-dev-pipeline</action>
|
||||
<action>Parameters: mode=batch, story_key={{story_key}}</action>
|
||||
<action>Parameters: mode=batch, story_key={{story_key}}, complexity_level={{story_key}}.complexity.level</action>
|
||||
|
||||
<check if="super-dev-pipeline succeeded">
|
||||
<output>✅ Implementation complete: {{story_key}}</output>
|
||||
|
|
@ -507,7 +527,7 @@ Spawning Task agents in parallel...
|
|||
CRITICAL INSTRUCTIONS:
|
||||
1. Load workflow.xml: _bmad/core/tasks/workflow.xml
|
||||
2. Load workflow config: _bmad/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml
|
||||
3. Execute in BATCH mode with story_key={{story_key}}
|
||||
3. Execute in BATCH mode with story_key={{story_key}} and complexity_level={{story_key}}.complexity.level
|
||||
4. Follow all 7 pipeline steps (init, pre-gap, implement, post-validate, code-review, complete, summary)
|
||||
5. Commit changes when complete
|
||||
6. Report final status (done/failed) with file list
|
||||
|
|
|
|||
|
|
@ -61,6 +61,33 @@ complexity:
|
|||
medium: ["api", "integration", "external", "third-party", "cache"]
|
||||
low: ["ui", "style", "config", "docs", "test"]
|
||||
|
||||
# Keyword matching configuration (defines how risk keywords are detected)
|
||||
keyword_matching:
|
||||
case_sensitive: false # "AUTH" matches "auth"
|
||||
require_word_boundaries: true # "auth" won't match "author"
|
||||
match_strategy: "exact" # exact word match required (no stemming)
|
||||
scan_locations:
|
||||
- story_title
|
||||
- task_descriptions
|
||||
- subtask_descriptions
|
||||
# Keyword variants (synonyms that map to canonical forms)
|
||||
variants:
|
||||
auth: ["authentication", "authorize", "authorization", "authz", "authn"]
|
||||
database: ["db", "databases", "datastore"]
|
||||
payment: ["payments", "pay", "billing", "checkout"]
|
||||
migration: ["migrations", "migrate"]
|
||||
security: ["secure", "security"]
|
||||
encryption: ["encrypt", "encrypted", "cipher"]
|
||||
|
||||
# Task counting rules
|
||||
task_counting:
|
||||
method: "top_level_only" # Only count [ ] at task level, not subtasks
|
||||
# Options: "top_level_only", "include_subtasks", "weighted"
|
||||
# Example:
|
||||
# - [ ] Parent task <- counts as 1
|
||||
# - [ ] Subtask 1 <- ignored
|
||||
# - [ ] Subtask 2 <- ignored
|
||||
|
||||
# Execution settings
|
||||
execution:
|
||||
continue_on_failure: true # Keep processing remaining stories if one fails
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
name: super-dev-pipeline
|
||||
description: "Step-file architecture with complexity-based routing and smart batching. Micro stories get lightweight path, standard/complex get full quality gates."
|
||||
author: "BMad"
|
||||
version: "1.2.0" # Added complexity-based routing
|
||||
version: "1.3.0" # Synchronized with batch-super-dev for complexity-based routing
|
||||
|
||||
# Critical variables from config
|
||||
config_source: "{project-root}/_bmad/bmm/config.yaml"
|
||||
|
|
|
|||
Loading…
Reference in New Issue