diff --git a/ASSESSMENT_SUMMARY.txt b/ASSESSMENT_SUMMARY.txt new file mode 100644 index 00000000..5afd5a64 --- /dev/null +++ b/ASSESSMENT_SUMMARY.txt @@ -0,0 +1,184 @@ +================================================================================ +DOCUMENTATION ASSESSMENT SUMMARY - v1.3.0 Complexity Routing +================================================================================ + +Assessment Date: 2026-01-07 +Commit: e5ede9ec (feat: add complexity-based routing and pipeline optimizations) +Files Analyzed: 9 changed files across batch-super-dev, super-dev-pipeline, dev-story + +================================================================================ +OVERALL FINDINGS +================================================================================ + +✅ STRENGTHS: + - Clear version markers (mostly consistent) + - Well-structured step-file architecture + - Consistent tone and professional style + - Good use of emoji for visual hierarchy + +⚠️ ISSUES FOUND: + - 3 CRITICAL issues (blocking implementation) + - 2 HIGH priority issues (quality gates) + - 3 MEDIUM issues (consistency) + - Zero LOW priority issues requiring action + +QUALITY SCORE: 7/10 + +================================================================================ +CRITICAL ISSUES (Must Fix Before Merge) +================================================================================ + +1. TASK COUNTING METHOD UNDEFINED (Line 230) + File: batch-super-dev/instructions.md + Impact: Complexity scoring inconsistent across stories + Fix: Add explicit definition including nested subtasks handling + +2. FILE_COUNT COLLECTED BUT UNUSED (Line 231) + File: batch-super-dev/instructions.md + Impact: Dead code, confusing developers + Fix: Either remove collection OR implement in complexity formula + +3. OVERLAPPING COMPLEXITY THRESHOLDS (Lines 242-244) + File: batch-super-dev/instructions.md + Impact: Edge cases route incorrectly + Example: 4-task "security" story with HIGH keyword + Fix: Replace with explicit decision tree (see SUGGESTED_FIXES.md) + +================================================================================ +HIGH PRIORITY ISSUES (Quality & Integration) +================================================================================ + +4. MULTI-AGENT MERGE STRATEGY MISSING (Lines 38-74) + File: step-05-code-review.md + Impact: Different merge results depending on implementer + Fix: Document deduplication, precedence, and merge algorithm + +5. RISK KEYWORD MATCHING UNDEFINED (Line 232) + File: batch-super-dev/instructions.md + Impact: ±5 points variance in complexity scoring + Issues: Case sensitivity, substring vs whole-word, deduplication + Fix: Specify exact matching rules with pseudocode + +================================================================================ +VERSION & CONSISTENCY ISSUES +================================================================================ + +6. VERSION MARKER MISMATCH (Line 94) + File: step-01-init.md + Current: "NEW v1.2.0" + Should: "NEW v1.3.0" (to match commit and batch-super-dev) + +7. BAILOUT vs SKIP TERMINOLOGY (Sections 4.5 + 6) + File: step-01-init.md + Issue: Both use similar language but mean different things + Fix: Document distinction clearly (see SUGGESTED_FIXES.md) + +8. CASE INCONSISTENCY (Multiple files) + Issue: "MICRO" vs "micro" used inconsistently + Fix: Document convention (uppercase display, lowercase storage) + +================================================================================ +DECISION TREE EXAMPLE (FIX #3) +================================================================================ + +Current (Problematic): + MICRO: task_count ≤ 3 AND complexity_score ≤ 5 AND no HIGH risk + COMPLEX: task_count ≥ 16 OR complexity_score ≥ 20 OR has HIGH risk + STANDARD: everything else + +Proposed (Clear & Unambiguous): + Step 1: Has HIGH keyword? → COMPLEX (STOP) + Step 2: task_count ≥ 16 OR score ≥ 20? → COMPLEX (STOP) + Step 3: task_count ≤ 3 AND score ≤ 5? → MICRO (STOP) + Step 4: Default → STANDARD + +Test Cases: + ✓ 2-task UI story: MICRO + ✓ 4-task auth story: COMPLEX (HIGH keyword) + ✓ 8-task standard story: STANDARD + ✓ 15-task database migration: COMPLEX (HIGH keywords) + +================================================================================ +DOCUMENTATION DELIVERABLES +================================================================================ + +1. DOCUMENTATION_ASSESSMENT.md (This analysis) + - Full assessment with detailed findings + - Quality ratings and metrics + - Test scenarios for complexity scoring + - 11 sections covering all aspects + +2. CRITICAL_FIXES_REQUIRED.md + - Blocking issues only + - Business impact explanation + - Specific file paths and line numbers + - Sign-off checklist before merge + +3. SUGGESTED_FIXES.md + - Exact replacement text for all 5 critical issues + - Code examples and pseudocode + - Complete working solutions ready to copy-paste + - 9 detailed fixes with implementation examples + +================================================================================ +RECOMMENDED ACTION PLAN +================================================================================ + +PHASE 1: Address Critical Issues (2-3 hours) + ☐ Fix task counting method (FIX #1) + ☐ Decide on file_count (FIX #2) + ☐ Implement decision tree (FIX #3) + ☐ Document merge strategy (FIX #4) + ☐ Specify keyword matching (FIX #5) + +PHASE 2: Fix Versions & Consistency (30 minutes) + ☐ Update version markers (FIX #6) + ☐ Document bailout/skip (FIX #7) + ☐ Document case convention (FIX #8) + +PHASE 3: QA & Testing (1-2 hours) + ☐ Test complexity scoring edge cases + ☐ Verify template rendering + ☐ Review decision tree logic + ☐ Check all file paths are absolute + +PHASE 4: Review & Sign-Off (1 hour) + ☐ PR review with focus on critical fixes + ☐ Merge with confidence + +Total Estimate: 4-6 hours + +================================================================================ +KEY METRICS +================================================================================ + +Clarity: 7/10 (Good, but some ambiguous sections) +Consistency: 7/10 (Mostly consistent, some case/version issues) +Completeness: 6/10 (Missing integration details) +Correctness: 6/10 (Critical logic issues in thresholds) + +Files Affected: 4 core files +Lines Modified: ~150 lines across all fixes +Breaking Changes: None (backward compatible) +Risk Level: MEDIUM (issues are fixable, non-fundamental) + +================================================================================ +CONTACT & REFERENCES +================================================================================ + +Commit: e5ede9ec +Feature: Complexity-Based Routing v1.3.0 +Assessment: 2026-01-07 + +Files to Review: +- src/modules/bmm/workflows/4-implementation/batch-super-dev/instructions.md +- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-01-init.md +- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-03-implement.md +- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-04-post-validation.md +- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/steps/step-05-code-review.md +- src/modules/bmm/workflows/4-implementation/batch-super-dev/workflow.yaml +- src/modules/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml + +================================================================================ +END OF SUMMARY +================================================================================ diff --git a/src/modules/bmm/workflows/4-implementation/batch-super-dev/instructions.md b/src/modules/bmm/workflows/4-implementation/batch-super-dev/instructions.md index 080f0339..4ec0d1d9 100644 --- a/src/modules/bmm/workflows/4-implementation/batch-super-dev/instructions.md +++ b/src/modules/bmm/workflows/4-implementation/batch-super-dev/instructions.md @@ -227,21 +227,41 @@ Run `/bmad:bmm:workflows:sprint-status` to see status. Read story file: {{file_path}} - Count unchecked tasks ([ ]) in Tasks/Subtasks section → task_count + Count unchecked tasks ([ ]) at top level only in Tasks/Subtasks section → task_count + (See workflow.yaml complexity.task_counting.method = "top_level_only") + Extract file paths mentioned in tasks → file_count - Scan story title and task descriptions for risk keywords + Scan story title and task descriptions for risk keywords using rules from workflow.yaml: + - Case insensitive matching (require_word_boundaries: true) + - Include keyword variants (e.g., "authentication" matches "auth") + - Scan: story_title, task_descriptions, subtask_descriptions + Calculate complexity score: - Base score = task_count - - Add 5 for each HIGH risk keyword (auth, security, payment, migration, database, schema) - - Add 2 for each MEDIUM risk keyword (api, integration, external, cache) + - Add 5 for each HIGH risk keyword match (auth, security, payment, migration, database, schema, encryption) + - Add 2 for each MEDIUM risk keyword match (api, integration, external, third-party, cache) - Add 0 for LOW risk keywords (ui, style, config, docs, test) + - Count each keyword only once (no duplicates) - Assign complexity level: - - MICRO: task_count ≤ 3 AND complexity_score ≤ 5 AND no HIGH risk keywords - - COMPLEX: task_count ≥ 16 OR complexity_score ≥ 20 OR has HIGH risk keywords - - STANDARD: everything else + Assign complexity level using mutually exclusive decision tree (priority order): + + 1. Check COMPLEX first (highest priority): + IF (task_count ≥ 16 OR complexity_score ≥ 20 OR has ANY HIGH risk keyword) + THEN level = COMPLEX + + 2. Else check MICRO (lowest complexity): + ELSE IF (task_count ≤ 3 AND complexity_score ≤ 5 AND file_count ≤ 5) + THEN level = MICRO + + 3. Else default to STANDARD: + ELSE level = STANDARD + + This ensures no overlaps: + - Story with HIGH keyword → COMPLEX (never MICRO or STANDARD) + - Story with 4-15 tasks or >5 files → STANDARD (not MICRO or COMPLEX) + - Story with ≤3 tasks, ≤5 files, no HIGH keywords → MICRO Store complexity_level for story: {{story_key}}.complexity = {level, score, task_count, risk_keywords} @@ -418,7 +438,7 @@ Only the first {{max_stories}} will be processed. Invoke workflow: /bmad:bmm:workflows:super-dev-pipeline - Parameters: mode=batch, story_key={{story_key}} + Parameters: mode=batch, story_key={{story_key}}, complexity_level={{story_key}}.complexity.level ✅ Implementation complete: {{story_key}} @@ -507,7 +527,7 @@ Spawning Task agents in parallel... CRITICAL INSTRUCTIONS: 1. Load workflow.xml: _bmad/core/tasks/workflow.xml 2. Load workflow config: _bmad/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml - 3. Execute in BATCH mode with story_key={{story_key}} + 3. Execute in BATCH mode with story_key={{story_key}} and complexity_level={{story_key}}.complexity.level 4. Follow all 7 pipeline steps (init, pre-gap, implement, post-validate, code-review, complete, summary) 5. Commit changes when complete 6. Report final status (done/failed) with file list diff --git a/src/modules/bmm/workflows/4-implementation/batch-super-dev/workflow.yaml b/src/modules/bmm/workflows/4-implementation/batch-super-dev/workflow.yaml index 793ae25c..b93b2b9e 100644 --- a/src/modules/bmm/workflows/4-implementation/batch-super-dev/workflow.yaml +++ b/src/modules/bmm/workflows/4-implementation/batch-super-dev/workflow.yaml @@ -61,6 +61,33 @@ complexity: medium: ["api", "integration", "external", "third-party", "cache"] low: ["ui", "style", "config", "docs", "test"] + # Keyword matching configuration (defines how risk keywords are detected) + keyword_matching: + case_sensitive: false # "AUTH" matches "auth" + require_word_boundaries: true # "auth" won't match "author" + match_strategy: "exact" # exact word match required (no stemming) + scan_locations: + - story_title + - task_descriptions + - subtask_descriptions + # Keyword variants (synonyms that map to canonical forms) + variants: + auth: ["authentication", "authorize", "authorization", "authz", "authn"] + database: ["db", "databases", "datastore"] + payment: ["payments", "pay", "billing", "checkout"] + migration: ["migrations", "migrate"] + security: ["secure", "security"] + encryption: ["encrypt", "encrypted", "cipher"] + + # Task counting rules + task_counting: + method: "top_level_only" # Only count [ ] at task level, not subtasks + # Options: "top_level_only", "include_subtasks", "weighted" + # Example: + # - [ ] Parent task <- counts as 1 + # - [ ] Subtask 1 <- ignored + # - [ ] Subtask 2 <- ignored + # Execution settings execution: continue_on_failure: true # Keep processing remaining stories if one fails diff --git a/src/modules/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml b/src/modules/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml index 37fa2db2..ff5c28ad 100644 --- a/src/modules/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml +++ b/src/modules/bmm/workflows/4-implementation/super-dev-pipeline/workflow.yaml @@ -1,7 +1,7 @@ name: super-dev-pipeline description: "Step-file architecture with complexity-based routing and smart batching. Micro stories get lightweight path, standard/complex get full quality gates." author: "BMad" -version: "1.2.0" # Added complexity-based routing +version: "1.3.0" # Synchronized with batch-super-dev for complexity-based routing # Critical variables from config config_source: "{project-root}/_bmad/bmm/config.yaml"