From 5f183d29d5a5dea3ec804ade92bd4d14faf7239a Mon Sep 17 00:00:00 2001 From: Jonah Schulte Date: Fri, 26 Dec 2025 14:03:09 -0500 Subject: [PATCH] docs: add gap analysis user guide and migration docs - Complete gap analysis feature documentation - Migration guide for existing users - Usage examples for all planning styles - Troubleshooting and FAQ sections --- docs/gap-analysis-migration.md | 222 +++++++++++++++++++++ docs/gap-analysis.md | 352 +++++++++++++++++++++++++++++++++ 2 files changed, 574 insertions(+) create mode 100644 docs/gap-analysis-migration.md create mode 100644 docs/gap-analysis.md diff --git a/docs/gap-analysis-migration.md b/docs/gap-analysis-migration.md new file mode 100644 index 00000000..0c1b91c2 --- /dev/null +++ b/docs/gap-analysis-migration.md @@ -0,0 +1,222 @@ +# Gap Analysis Migration Guide + +## Quick Start + +**For existing BMAD users:** Gap analysis is now automatic. No changes required to your workflow! + +## What Changed + +### Before (BMAD v6.0.0-alpha.21 and earlier) + +``` +create-story → generates complete story with "verified" tasks +dev-story → blindly follows tasks as written +``` + +**Problem:** Tasks become stale in batch planning scenarios + +### After (BMAD v6.0.0-alpha.22+) + +``` +create-story → generates requirements + DRAFT tasks +dev-story → validates tasks against codebase → implementation +``` + +**Solution:** Tasks always reflect current codebase state + +## For Existing Stories + +### Stories Already Created + +**Option 1: No action needed** +- Gap analysis runs automatically when you use `dev-story` +- You'll be prompted to approve task updates +- Select [Y] to proceed with refinements + +**Option 2: Regenerate stories** (optional) +- Delete existing story files +- Run `create-story` again +- New stories will have "DRAFT TASKS" notation + +### Stories In Progress + +**If you've already started development:** +- Gap analysis detects partial implementations +- Proposes tasks for remaining work only +- Use [e] Edit to manually adjust if needed + +## New Workflow Steps + +### When Planning (create-story) + +**No changes to your workflow!** Just be aware: +- Tasks are now marked as "DRAFT" +- They'll be validated at dev-time +- This is expected behavior + +### When Developing (dev-story) + +**New Step 1.5 (automatic):** + +1. Gap analysis runs +2. Shows findings (What Exists / What's Missing) +3. Proposes task updates +4. Waits for your approval + +**Your options:** +- [Y] Yes - approve and proceed (recommended) +- [A] Auto-accept - apply + auto-accept future (automation) +- [n] No - keep draft tasks (not recommended) +- [e] Edit - manually adjust +- [s] Skip - halt development +- [r] Review - more details first + +## Common Scenarios + +### Scenario 1: Just-in-Time Planning + +**Before:** +```bash +/create-story # Story 1.1 +/dev-story # Implement immediately +``` + +**After:** +```bash +/create-story # Story 1.1 with DRAFT tasks +/dev-story # Gap analysis → minimal changes → implement +``` + +**Impact:** Minimal. Tasks stay mostly accurate. + +### Scenario 2: Batch Planning (This is the game-changer!) + +**Before:** +```bash +Day 1: +/create-story # Story 1.1 says "Create validation.ts" +/create-story # Story 1.2 says "Create validation.ts" (duplicate!) + +Day 2: +/dev-story # Implements 1.1, creates validation.ts +/dev-story # Implements 1.2, creates validation.ts AGAIN ❌ +``` + +**After:** +```bash +Day 1: +/create-story # Story 1.1 DRAFT: "Create validation.ts" +/create-story # Story 1.2 DRAFT: "Create validation.ts" + +Day 2: +/dev-story # Gap analysis: nothing exists → creates validation.ts ✅ +/dev-story # Gap analysis: validation.ts exists! + # Updated task: "Extend validation.ts" ✅ +``` + +**Impact:** Massive! Prevents all duplicate implementations. + +## For Custom Workflows + +### If You've Customized create-story + +**Changes you need to merge:** +1. Step labels changed (removed "EXHAUSTIVE ANALYSIS") +2. Focus shifted to requirements (not codebase scanning) +3. Tasks marked as "DRAFT" in story files + +**Compatibility:** High. Changes are mostly textual. + +### If You've Customized dev-story + +**Changes you need to merge:** +1. New Step 1.5 (gap analysis) between Step 1 and 2 +2. Step numbers shifted (old Step 2 is now Step 2, but Step 1.5 added) +3. Story file updates (Gap Analysis section, Change Log) + +**Compatibility:** Medium. Review carefully if you have custom logic. + +### If You've Customized Both + +**Recommendation:** +1. Review changes in both workflows +2. Test with a sample story +3. Merge your customizations carefully +4. Validate with your use cases + +## Rollback Instructions + +If gap analysis causes issues, you can temporarily revert: + +### Temporary Workaround (keep updated workflows) + +When dev-story prompts for gap analysis approval: +- Select [n] No to keep original draft tasks +- Development proceeds without task refinement + +### Full Rollback (revert to old workflows) + +```bash +# In BMAD-METHOD repo +git checkout v6.0.0-alpha.21 -- src/modules/bmm/workflows/4-implementation/ +git checkout v6.0.0-alpha.21 -- src/modules/bmgd/workflows/4-production/ + +# Reinstall in your project +cd ~/git/ai/platform +rm -rf _bmad +npx bmad-method@alpha install +``` + +**Note:** Report issues to help us improve gap analysis! + +## Testing Checklist + +Before fully adopting gap analysis, test these scenarios: + +- [ ] **Just-in-time planning** - Create + develop story immediately +- [ ] **Batch planning** - Create 3-5 stories, develop sequentially +- [ ] **Auto-accept mode** - Use [A] option for automation +- [ ] **Edit mode** - Use [e] to manually adjust tasks +- [ ] **Existing codebase** - Develop story in project with existing code +- [ ] **Greenfield project** - Develop first story (nothing exists) + +## FAQ + +### Q: Do I need to regenerate existing stories? + +**A:** No. Gap analysis works with any story format. It will validate tasks regardless of when the story was created. + +### Q: Will this break my CI/CD pipeline? + +**A:** Only if your pipeline auto-runs dev-story. Add [A] auto-accept flag or pre-approve in your automation. + +### Q: Can I keep the old behavior? + +**A:** Select [n] when prompted. But you'll lose the benefits of gap analysis. + +### Q: What if gap analysis is wrong? + +**A:** Use [e] Edit to manually correct, or [n] to keep original tasks. Also report the issue so we can improve detection. + +## Getting Help + +**Issues or questions?** +1. Join [Discord #general-dev](https://discord.gg/gk8jAdXWmj) +2. Open issue in [BMAD-METHOD repo](https://github.com/bmad-code-org/BMAD-METHOD) +3. Check [full gap analysis documentation](./gap-analysis.md) + +## Feedback Welcome + +This is a new feature in BMAD v6. We'd love your feedback! + +**What we want to know:** +- Does gap analysis work correctly for your use case? +- Are the proposed task updates accurate? +- Any false positives/negatives in codebase scanning? +- Feature requests or improvements? + +Share in Discord or open a GitHub issue. + +--- + +**Gap Analysis: Keeping BMAD stories grounded in reality since v6.0.0-alpha.22** diff --git a/docs/gap-analysis.md b/docs/gap-analysis.md new file mode 100644 index 00000000..2a08fad2 --- /dev/null +++ b/docs/gap-analysis.md @@ -0,0 +1,352 @@ +# Gap Analysis: Codebase-Reality Task Validation + +## Overview + +Gap Analysis is a critical feature in BMAD v6 that ensures story implementation tasks accurately reflect the current state of your codebase. By scanning your project before development begins, it prevents duplicate implementations, identifies missing dependencies, and adapts tasks to leverage existing code. + +## The Problem + +### Before Gap Analysis + +**Batch Planning Scenario:** +``` +Day 1: Plan all stories +- Story 1.1: "Create validation.ts with EmailValidator" +- Story 1.2: "Create user service using validation" +- Story 1.3: "Add signup with validation" + +Day 2: Develop Story 1.1 +- Creates validation.ts ✅ + +Day 3: Develop Story 1.2 +- Story still says "Create validation.ts" +- Dev agent creates it AGAIN ❌ +- Now have duplicate code +``` + +**Result:** Wasted time, duplicate implementations, confusion about which code to use. + +### The Root Cause + +Stories planned ahead of time become **stale** the moment you implement earlier stories. The codebase evolves, but static story files don't adapt. + +## The Solution + +Gap Analysis happens at **dev-time** (when `dev-story` runs) to ensure tasks reflect **current codebase reality**. + +### How It Works + +``` +┌─────────────────────────────────────────────┐ +│ create-story │ +│ • Requirements analysis │ +│ • DRAFT tasks (requirements-based) │ +│ • Status: ready-for-dev │ +└─────────────────┬───────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────┐ +│ dev-story │ +│ Step 1.5: GAP ANALYSIS │ +│ • Scan codebase (Glob/Grep/Read) │ +│ • Document What Exists vs What's Missing │ +│ • Propose task refinements │ +│ • Get user approval │ +│ • Update story with refined tasks │ +└─────────────────┬───────────────────────────┘ + │ + ▼ + Implementation begins + with ACCURATE tasks +``` + +## Usage + +### Standard Workflow + +1. **Plan Stories** (same as before): + ```bash + /create-story + ``` + - Generates requirements, acceptance criteria + - Creates **DRAFT tasks** based on requirements + - Status: `ready-for-dev` + +2. **Develop Story**: + ```bash + /dev-story + ``` + - Step 1.5 runs **automatically** + - Scans codebase + - Shows findings and proposed task updates + - Asks for approval + +### Approval Options + +When gap analysis presents findings, you have 6 options: + +| Option | Description | When to Use | +|--------|-------------|-------------| +| **[Y]** Yes | Approve and proceed | Trust the analysis | +| **[A]** Auto-accept | Apply + auto-accept future refinements | Full automation mode | +| **[n]** No | Keep draft tasks | You know better than the scan | +| **[e]** Edit | Manually adjust proposed tasks | Want custom changes | +| **[s]** Skip | Halt development | Something looks wrong | +| **[r]** Review | Deeper inspection first | Need more details | + +### Example Session + +``` +📊 Gap Analysis Complete + +Codebase Scan Results: + +✅ What Exists: +- src/lib/validation.ts (EmailValidator implemented) +- src/api/user.routes.ts (basic CRUD endpoints) +- 15 existing test files with 82% coverage + +❌ What's Missing: +- No authentication service layer +- No PhoneValidator +- No auth-related tests + +--- + +📝 Proposed Task Updates: + +ADDED TASKS (new prerequisites discovered): ++ Create src/services/auth.service.ts (dependency found) + +MODIFIED TASKS (adjusted for codebase reality): +- [ORIGINAL] Create src/lib/validation.ts ++ [UPDATED] Extend validation.ts with PhoneValidator (file exists) + +REMOVED TASKS (already complete): +× Setup testing framework (jest.config.js exists) + +Approve these task updates? [Y/A/n/e/s/r] +``` + +## Auto-Accept Mode + +For full automation, use **[A] Auto-accept** option: + +```bash +User: "Yes, just auto-accept whatever tasks you discover" +``` + +Benefits: +- No repeated prompts for remaining stories +- All refinements applied automatically +- Still documented in Change Log +- Perfect for CI/CD or batch processing + +## Planning Styles Supported + +### 1. Just-in-Time Planning + +``` +Day 1: Plan Story 1.1 → Develop Story 1.1 +Day 2: Plan Story 1.2 → Develop Story 1.2 +``` + +**Gap Analysis Impact:** Minimal changes (tasks stay mostly accurate) + +### 2. Batch Planning + +``` +Day 1: Plan Stories 1.1, 1.2, 1.3, 1.4, 1.5 +Day 2-5: Develop all stories sequentially +``` + +**Gap Analysis Impact:** Critical (prevents duplicate code from stale plans) + +### 3. Mid-Sprint Planning + +``` +Week 1: Stories 1.1-1.3 completed +Week 2: Plan Stories 1.4-1.7 (codebase has evolved) +``` + +**Gap Analysis Impact:** Adapts to new foundation built by earlier stories + +## Story File Updates + +### Draft Tasks (from create-story) + +```markdown +## Tasks / Subtasks + +⚠️ **DRAFT TASKS** - Generated from requirements analysis. +Will be validated and refined against actual codebase when dev-story runs. + +- [ ] Create src/lib/validation.ts +- [ ] Implement EmailValidator +- [ ] Write validator tests +``` + +### After Gap Analysis (from dev-story) + +```markdown +## Gap Analysis + +Scanned: 2025-01-18 at 10:00 AM + +✅ What Exists: +- src/lib/validation.ts (EmailValidator) +- 15 test files with 82% coverage + +❌ What's Missing: +- No PhoneValidator +- No validation tests for phone numbers + +Task Changes Applied: +- MODIFIED: "Create validation.ts" → "Extend validation.ts with PhoneValidator" +- MODIFIED: "Write tests" → "Add PhoneValidator tests to validation.test.ts" +- REMOVED: "Set up testing framework" (already configured) + +--- + +## Tasks / Subtasks + +- [ ] Extend src/lib/validation.ts with PhoneValidator +- [ ] Add PhoneValidator tests to validation.test.ts +``` + +## Benefits + +### 1. Prevents Duplicate Code +- Scans before creating files +- Suggests "extend" instead of "create" +- Identifies reusable code + +### 2. Discovers Missing Dependencies +- Identifies prerequisites stories assumed exist +- Adds missing tasks automatically +- Prevents mid-implementation blockers + +### 3. Removes Completed Work +- Detects features already implemented +- Removes redundant tasks +- Saves development time + +### 4. Works for All Planning Styles +- Just-in-time: Validates assumptions +- Batch planning: Corrects staleness +- Mid-sprint: Adapts to evolved codebase + +### 5. Maintains Story Accuracy +- Documents scan results +- Updates Change Log +- Preserves decision history + +## Migration Guide + +### Existing Projects + +If you have stories created before this feature: + +1. **No action required** - gap analysis runs automatically when you use `dev-story` +2. **First story per epic** will perform full analysis +3. **User approval required** before tasks change +4. **Opt-out available** with [n] option if needed + +### For Custom Workflows + +If you've customized `create-story` or `dev-story`: + +1. **Merge carefully** - gap analysis is in Step 1.5 +2. **Preserve custom logic** - gap analysis is self-contained +3. **Test thoroughly** - validate with your customizations + +## Technical Details + +### Scan Targets + +Gap analysis uses: +- **Glob** - Find files matching patterns (`**/*.ts`, `**/*.test.ts`) +- **Grep** - Search for classes, functions, components +- **Read** - Verify implementation details + +### Performance + +- Typical scan: **5-10 seconds** for medium codebase +- Runs **once per story** at dev-time +- No performance impact on `create-story` + +### Safety + +- **Read-only scans** - never modifies codebase +- **User approval required** - no automatic rewrites +- **Documented changes** - all modifications tracked +- **Reversible** - can reject refinements + +## Troubleshooting + +### "Tasks keep changing every time I run dev-story" + +**Cause:** Codebase evolving between attempts +**Solution:** This is expected! Gap analysis adapts to current state. If unwanted, use [n] to keep original tasks. + +### "Gap analysis removed a task I need" + +**Cause:** Scan detected the task is already complete +**Solution:** Use [e] Edit option to manually add the task back, or verify the feature truly exists. + +### "Scan found files but doesn't recognize they're complete" + +**Cause:** Implementation doesn't match task description +**Solution:** Gap analysis is conservative. Use [e] Edit to refine, or [n] to keep original tasks. + +### "I want to skip gap analysis entirely" + +**Not recommended**, but possible: +- Select [n] No to keep draft tasks +- Risk: Duplicate implementations, wasted time + +## FAQ + +### Q: Does gap analysis slow down development? + +**A:** Minimal impact (5-10 seconds). Prevents hours of duplicate work and debugging. + +### Q: Can I disable gap analysis? + +**A:** Select [n] when prompted. Not recommended for batch planning. + +### Q: What if scan misses something? + +**A:** Use [e] Edit to manually adjust tasks before proceeding. + +### Q: Does this work with custom workflows? + +**A:** Yes. Gap analysis is self-contained in Step 1.5 of dev-story. + +### Q: What about greenfield projects? + +**A:** Gap analysis runs but finds nothing (instant). Confirms blank slate. No harm done. + +### Q: Can I auto-accept for just this session? + +**A:** Yes! Use [A] Auto-accept. Preference doesn't persist across sessions. + +## Best Practices + +1. **Trust the analysis** - It's based on actual code scans, not guesses +2. **Use auto-accept for known-good repos** - Speeds up batch development +3. **Review findings on first use** - Understand what gap analysis detects +4. **Edit when needed** - You know your codebase better than scans +5. **Don't skip for batch planning** - This is when gap analysis shines + +## Contributing + +Found an issue or want to improve gap analysis? + +1. Open an issue in [BMAD-METHOD repo](https://github.com/bmad-code-org/BMAD-METHOD) +2. Discuss in Discord #general-dev +3. Submit PR following [contribution guidelines](../CONTRIBUTING.md) + +--- + +**Gap Analysis ensures BMAD stories reflect reality, not aspirations.**