docs(batch-super-dev): Document agent limitations and fix workflow issues
Issue discovered in platform project batch execution: - Attempted to batch-regenerate 14 skeleton stories - Agents cannot invoke /create-story-with-gap-analysis workflow - Batch execution requires pre-generated BMAD stories ROOT CAUSE: - Agents cannot execute slash commands (user CLI only) - Agents cannot invoke other BMAD workflows autonomously - Story creation requires interactive user input IMPROVEMENTS: 1. AGENT-LIMITATIONS.md - Clear documentation of what agents can/cannot do 2. WORKFLOW-PATCH-STEP-2.5.md - Fix for batch-super-dev Step 2.5 3. BMAD-WORKFLOW-IMPROVEMENTS.md - Complete analysis and recommendations KEY LESSON: Batch-super-dev is for EXECUTION, not CREATION. Story generation must happen before batch execution. Files: - src/modules/bmm/workflows/4-implementation/batch-super-dev/AGENT-LIMITATIONS.md - src/modules/bmm/workflows/4-implementation/batch-super-dev/WORKFLOW-PATCH-STEP-2.5.md - BMAD-WORKFLOW-IMPROVEMENTS.md - src/modules/cis/module.yaml (prettier formatting)
This commit is contained in:
Jonah Schulte
parent
0d66a28cfe
commit
80ec065117
|
|
@ -0,0 +1,395 @@
|
||||||
|
# BMAD Workflow Improvements - Lessons from Batch Execution
|
||||||
|
|
||||||
|
**Date:** 2026-01-08
|
||||||
|
**Context:** Batch-super-dev tried to regenerate 14 skeleton stories but failed
|
||||||
|
**Root Cause:** Agents cannot invoke workflows/skills in batch mode
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Issue Summary
|
||||||
|
|
||||||
|
**What Happened:**
|
||||||
|
1. Created 21 skeleton sub-story files (just headers + widget lists)
|
||||||
|
2. Launched 14 agents via batch-super-dev to process them
|
||||||
|
3. Agents correctly identified stories were incomplete (no tasks section)
|
||||||
|
4. Agents correctly halted per super-dev-pipeline Step 1.4.5
|
||||||
|
5. BUT: Batch-super-dev Step 2.5 says to "Invoke workflow: /bmad..." which agents can't do
|
||||||
|
6. Result: 13/14 agents halted correctly, 1 somehow succeeded
|
||||||
|
|
||||||
|
**The Problem:**
|
||||||
|
- Workflows correctly validate and halt
|
||||||
|
- BUT batch-super-dev suggests agents can regenerate stories
|
||||||
|
- Agents CANNOT execute `/` commands or invoke workflows autonomously
|
||||||
|
- This creates false expectations
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Proposed Fixes
|
||||||
|
|
||||||
|
### Fix 1: Update batch-super-dev Step 2.5 (Story Validation)
|
||||||
|
|
||||||
|
**Current Code (lines 82-99):**
|
||||||
|
```xml
|
||||||
|
<ask>Create story file with gap analysis? (yes/no):</ask>
|
||||||
|
|
||||||
|
<check if="response == 'yes'">
|
||||||
|
<output>Creating story {{story_key}} with codebase gap analysis...</output>
|
||||||
|
<action>Invoke workflow: /bmad:bmm:workflows:create-story-with-gap-analysis</action>
|
||||||
|
<action>Parameters: story_key={{story_key}}</action>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
- "Invoke workflow: /bmad..." doesn't work for agents
|
||||||
|
- Agents can't run slash commands
|
||||||
|
- This will always fail in batch mode
|
||||||
|
|
||||||
|
**Recommended Fix:**
|
||||||
|
```xml
|
||||||
|
<ask>Create story file with gap analysis? (yes/no):</ask>
|
||||||
|
|
||||||
|
<check if="response == 'yes'">
|
||||||
|
<check if="batch mode">
|
||||||
|
<output>
|
||||||
|
⚠️ CANNOT CREATE STORIES IN BATCH MODE
|
||||||
|
|
||||||
|
Agents cannot invoke workflows autonomously.
|
||||||
|
|
||||||
|
**Required Action:**
|
||||||
|
1. Exit batch-super-dev
|
||||||
|
2. Run manually: /create-story-with-gap-analysis
|
||||||
|
3. Specify story key: {{story_key}}
|
||||||
|
4. Re-run batch-super-dev after story created
|
||||||
|
|
||||||
|
Skipping story {{story_key}} from batch.
|
||||||
|
</output>
|
||||||
|
<action>Mark story for removal from selection</action>
|
||||||
|
<action>Add to skipped_stories with reason: "Story creation requires manual workflow"</action>
|
||||||
|
</check>
|
||||||
|
|
||||||
|
<check if="interactive mode">
|
||||||
|
<output>Creating story {{story_key}} with codebase gap analysis...</output>
|
||||||
|
<output>
|
||||||
|
⚠️ IMPORTANT: Story creation requires /create-story-with-gap-analysis workflow
|
||||||
|
|
||||||
|
This workflow cannot be invoked from within another workflow.
|
||||||
|
|
||||||
|
**Please run:**
|
||||||
|
```
|
||||||
|
/create-story-with-gap-analysis
|
||||||
|
```
|
||||||
|
|
||||||
|
Then return to this workflow.
|
||||||
|
</output>
|
||||||
|
<action>HALT - Cannot auto-invoke create-story workflow</action>
|
||||||
|
</check>
|
||||||
|
</check>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why This Works:**
|
||||||
|
- ✅ Explicitly states agents can't create stories in batch
|
||||||
|
- ✅ Tells user exactly what to do
|
||||||
|
- ✅ Prevents false expectations
|
||||||
|
- ✅ Provides clear halt message
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Fix 2: Add Agent Limitations Section to batch-super-dev
|
||||||
|
|
||||||
|
**Add new section after line 5:**
|
||||||
|
|
||||||
|
```xml
|
||||||
|
<workflow>
|
||||||
|
|
||||||
|
<agent-limitations critical="true">
|
||||||
|
<limitation n="1">
|
||||||
|
<title>Agents Cannot Invoke Workflows</title>
|
||||||
|
<description>
|
||||||
|
Task agents running in batch mode cannot execute slash commands (/)
|
||||||
|
or invoke other BMAD workflows. If a story requires creation or
|
||||||
|
regeneration, the batch must halt and user must handle manually.
|
||||||
|
</description>
|
||||||
|
<impact>
|
||||||
|
Stories without proper BMAD format (12 sections, tasks) will be skipped
|
||||||
|
in batch execution. User must run /create-story-with-gap-analysis first.
|
||||||
|
</impact>
|
||||||
|
</limitation>
|
||||||
|
|
||||||
|
<limitation n="2">
|
||||||
|
<title>Agents Cannot Prompt User Interactively</title>
|
||||||
|
<description>
|
||||||
|
Batch agents run autonomously. They cannot ask user questions mid-execution.
|
||||||
|
All decisions must be pre-configured or stories will be skipped.
|
||||||
|
</description>
|
||||||
|
<impact>
|
||||||
|
Optional steps are auto-skipped. Ambiguous requirements cause halt.
|
||||||
|
</impact>
|
||||||
|
</limitation>
|
||||||
|
|
||||||
|
<limitation n="3">
|
||||||
|
<title>Skeleton Stories Will Be Skipped</title>
|
||||||
|
<description>
|
||||||
|
If a story file has <12 BMAD sections or 0 tasks, super-dev-pipeline
|
||||||
|
will correctly halt with "Nothing to implement" message. This is NOT
|
||||||
|
a bug - it's correct behavior. User must regenerate story first.
|
||||||
|
</description>
|
||||||
|
<impact>
|
||||||
|
Batch execution requires ALL stories to be properly formatted BEFORE
|
||||||
|
running batch-super-dev. Do not attempt batch regeneration.
|
||||||
|
</impact>
|
||||||
|
</limitation>
|
||||||
|
</agent-limitations>
|
||||||
|
|
||||||
|
<step n="1" goal="Load and parse sprint-status.yaml">
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why This Helps:**
|
||||||
|
- ✅ Sets clear expectations upfront
|
||||||
|
- ✅ Explains why stories get skipped
|
||||||
|
- ✅ Prevents users from making same mistake I did
|
||||||
|
- ✅ Documents known limitations explicitly
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Fix 3: Enhance super-dev-pipeline Step 1.4.5 (Pre-Flight Bailout)
|
||||||
|
|
||||||
|
**Current Code (lines 94-105):**
|
||||||
|
```
|
||||||
|
If total_task_count == 0:
|
||||||
|
Display:
|
||||||
|
⚠️ EARLY BAILOUT: No Tasks Found
|
||||||
|
|
||||||
|
Story file exists but has no tasks in Tasks/Subtasks section.
|
||||||
|
- Story may be incomplete or malformed
|
||||||
|
- Run create-story or validate-create-story first
|
||||||
|
|
||||||
|
HALT - Nothing to implement
|
||||||
|
```
|
||||||
|
|
||||||
|
**Recommended Enhancement:**
|
||||||
|
```
|
||||||
|
If total_task_count == 0:
|
||||||
|
Display:
|
||||||
|
⚠️ EARLY BAILOUT: No Tasks Found
|
||||||
|
|
||||||
|
Story file exists but has no tasks in Tasks/Subtasks section.
|
||||||
|
|
||||||
|
**Common Causes:**
|
||||||
|
1. Story file is a skeleton/template (not generated by /create-story)
|
||||||
|
2. Story file is missing BMAD sections
|
||||||
|
3. Story file was manually created without proper format
|
||||||
|
|
||||||
|
**This is NOT a pipeline bug** - this is correct validation behavior.
|
||||||
|
|
||||||
|
**Required Action:**
|
||||||
|
{if batch mode}
|
||||||
|
- Story will be SKIPPED from batch execution
|
||||||
|
- User must regenerate: /create-story-with-gap-analysis
|
||||||
|
- Then re-run batch for this story
|
||||||
|
{else}
|
||||||
|
- Run: /create-story-with-gap-analysis
|
||||||
|
- Specify story key: {story_key}
|
||||||
|
- Then re-run super-dev-pipeline
|
||||||
|
{endif}
|
||||||
|
|
||||||
|
**IMPORTANT FOR AGENTS:**
|
||||||
|
Do NOT attempt to regenerate the story yourself. Agents cannot invoke
|
||||||
|
the /create-story-with-gap-analysis workflow. Document this issue and
|
||||||
|
halt gracefully.
|
||||||
|
|
||||||
|
HALT - Nothing to implement
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why This Helps:**
|
||||||
|
- ✅ Explains this is CORRECT behavior, not a bug
|
||||||
|
- ✅ Explicitly tells agents NOT to try regenerating
|
||||||
|
- ✅ Provides clear user action steps
|
||||||
|
- ✅ Different guidance for batch vs interactive mode
|
||||||
|
- ✅ Sets correct expectations
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Fix 4: Add Pre-Batch Validation Recommendations
|
||||||
|
|
||||||
|
**Add new document:** `_bmad/bmm/workflows/4-implementation/batch-super-dev/BATCH-BEST-PRACTICES.md`
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# Batch Super-Dev Best Practices
|
||||||
|
|
||||||
|
## Pre-Batch Checklist
|
||||||
|
|
||||||
|
**BEFORE running /batch-super-dev, verify:**
|
||||||
|
|
||||||
|
### ✅ All Story Files Exist
|
||||||
|
```bash
|
||||||
|
# Check all ready-for-dev stories have files
|
||||||
|
for story in $(grep "ready-for-dev" docs/sprint-artifacts/sprint-status.yaml | cut -d: -f1); do
|
||||||
|
if [ ! -f "docs/sprint-artifacts/story-$story.md" ]; then
|
||||||
|
echo "❌ Missing: $story"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ All Stories Have Proper BMAD Format
|
||||||
|
|
||||||
|
Run validation script:
|
||||||
|
```bash
|
||||||
|
./scripts/validate-bmad-format.sh docs/sprint-artifacts/story-*.md
|
||||||
|
```
|
||||||
|
|
||||||
|
**Required:** All stories must show "✅ All 12 sections present"
|
||||||
|
|
||||||
|
### ✅ All Stories Have Tasks
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check all stories have at least 1 task
|
||||||
|
for file in docs/sprint-artifacts/story-*.md; do
|
||||||
|
task_count=$(grep -c "^- \[ \]" "$file" 2>/dev/null || echo "0")
|
||||||
|
if [ "$task_count" -eq 0 ]; then
|
||||||
|
echo "❌ No tasks: $(basename $file)"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
**Required:** All stories must have unchecked tasks (something to implement)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Mistakes
|
||||||
|
|
||||||
|
### ❌ Mistake 1: Batch Regeneration
|
||||||
|
|
||||||
|
**Don't:**
|
||||||
|
```bash
|
||||||
|
# Create 20 skeleton story files
|
||||||
|
# Run batch-super-dev expecting agents to regenerate them
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why:** Agents cannot invoke /create-story-with-gap-analysis workflow
|
||||||
|
|
||||||
|
**Do Instead:**
|
||||||
|
```bash
|
||||||
|
# Regenerate stories one at a time
|
||||||
|
for story in story-20-13a-{1..5}; do
|
||||||
|
/create-story-with-gap-analysis # Interactive, per story
|
||||||
|
done
|
||||||
|
|
||||||
|
# THEN run batch-super-dev
|
||||||
|
/batch-super-dev
|
||||||
|
```
|
||||||
|
|
||||||
|
### ❌ Mistake 2: Mixed Story Quality
|
||||||
|
|
||||||
|
**Don't:**
|
||||||
|
- Mix properly generated stories with skeleton templates
|
||||||
|
- Assume agents will "fix" incomplete stories
|
||||||
|
- Launch batch without validating story format
|
||||||
|
|
||||||
|
**Do:**
|
||||||
|
- Validate ALL stories before batch
|
||||||
|
- Regenerate any incomplete ones
|
||||||
|
- Ensure consistent quality
|
||||||
|
|
||||||
|
### ❌ Mistake 3: Assuming Agents Can Run / Commands
|
||||||
|
|
||||||
|
**Don't:**
|
||||||
|
- Tell agents to "Run /create-story"
|
||||||
|
- Expect agents to invoke other workflows
|
||||||
|
- Assume agents can execute user CLI commands
|
||||||
|
|
||||||
|
**Do:**
|
||||||
|
- Pre-generate all stories before batch
|
||||||
|
- Use batch ONLY for implementation, not planning
|
||||||
|
- Keep story creation separate from story execution
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Success Patterns
|
||||||
|
|
||||||
|
### ✅ Pattern 1: Manual Create → Batch Execute
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Step 1: Create all stories manually (1-2 days)
|
||||||
|
/create-story-with-gap-analysis # For each of 20 stories
|
||||||
|
|
||||||
|
# Step 2: Validate all stories
|
||||||
|
./scripts/validate-all-stories.sh
|
||||||
|
|
||||||
|
# Step 3: Batch execute (parallel, fast)
|
||||||
|
/batch-super-dev # Select all 20 stories, run 4 agents
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ Pattern 2: Incremental Batch Execution
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Execute stories in small batches as you create them
|
||||||
|
/create-story-with-gap-analysis # Create 5 stories
|
||||||
|
|
||||||
|
/batch-super-dev # Execute those 5 (filter by epic)
|
||||||
|
|
||||||
|
/create-story-with-gap-analysis # Create 5 more
|
||||||
|
|
||||||
|
/batch-super-dev # Execute those 5
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ Pattern 3: Validation Before Batch
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Add to your workflow
|
||||||
|
validate_stories() {
|
||||||
|
for story in $@; do
|
||||||
|
if ! ./scripts/validate-bmad-format.sh "$story"; then
|
||||||
|
echo "❌ Story $story is not ready for batch"
|
||||||
|
echo "Run: /create-story-with-gap-analysis"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo "✅ All stories validated, ready for batch"
|
||||||
|
}
|
||||||
|
|
||||||
|
validate_stories docs/sprint-artifacts/story-20-13*.md
|
||||||
|
/batch-super-dev
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## When to Use Batch vs Individual
|
||||||
|
|
||||||
|
### Use Batch When:
|
||||||
|
- ✅ All stories properly formatted (12 sections, tasks present)
|
||||||
|
- ✅ Stories are independent (no complex dependencies)
|
||||||
|
- ✅ You want parallel execution (2-4 stories at once)
|
||||||
|
- ✅ Stories are similar complexity (all ~1 day effort)
|
||||||
|
|
||||||
|
### Use Individual (/dev-story) When:
|
||||||
|
- ✅ Story needs interactive decisions
|
||||||
|
- ✅ Story is complex (>3 days effort)
|
||||||
|
- ✅ Story has complex dependencies
|
||||||
|
- ✅ You want close monitoring/control
|
||||||
|
|
||||||
|
### DON'T Use Batch When:
|
||||||
|
- ❌ Stories are skeleton templates (not generated)
|
||||||
|
- ❌ Stories missing BMAD sections
|
||||||
|
- ❌ Mixed quality (some good, some incomplete)
|
||||||
|
- ❌ Need to create/regenerate stories
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Takeaway
|
||||||
|
|
||||||
|
**Batch-super-dev is for EXECUTION, not CREATION.**
|
||||||
|
|
||||||
|
Story creation is an interactive, context-heavy process that requires:
|
||||||
|
- User input on requirements
|
||||||
|
- Codebase scanning
|
||||||
|
- Gap analysis decisions
|
||||||
|
- Template generation
|
||||||
|
|
||||||
|
This cannot be reliably automated in batch mode. Always create stories first,
|
||||||
|
then batch execute them.
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Location:** `_bmad/bmm/workflows/4-implementation/batch-super-dev/BATCH-BEST-PRACTICES.md`
|
||||||
|
|
@ -0,0 +1,237 @@
|
||||||
|
# Agent Limitations in Batch Mode
|
||||||
|
|
||||||
|
**CRITICAL:** Agents running in batch-super-dev have specific limitations. Understanding these prevents wasted time and sets correct expectations.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core Limitations
|
||||||
|
|
||||||
|
### ❌ Agents CANNOT Invoke Other Workflows
|
||||||
|
|
||||||
|
**What this means:**
|
||||||
|
- Agents cannot run `/create-story-with-gap-analysis`
|
||||||
|
- Agents cannot execute `/` slash commands (those are for user CLI)
|
||||||
|
- Agents cannot call other BMAD workflows mid-execution
|
||||||
|
|
||||||
|
**Why:**
|
||||||
|
- Slash commands require user terminal context
|
||||||
|
- Workflow invocation requires special tool access
|
||||||
|
- Batch agents are isolated execution contexts
|
||||||
|
|
||||||
|
**Implication:**
|
||||||
|
- Story creation MUST happen before batch execution
|
||||||
|
- If stories are incomplete, batch will skip them
|
||||||
|
- No way to "fix" stories during batch
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ❌ Agents CANNOT Prompt User Interactively
|
||||||
|
|
||||||
|
**What this means:**
|
||||||
|
- Batch runs autonomously, no user interaction
|
||||||
|
- `<ask>` tags are auto-answered with defaults
|
||||||
|
- No way to clarify ambiguous requirements mid-batch
|
||||||
|
|
||||||
|
**Why:**
|
||||||
|
- Batch is designed for unattended execution
|
||||||
|
- User may not be present during execution
|
||||||
|
- Prompts would break parallel execution
|
||||||
|
|
||||||
|
**Implication:**
|
||||||
|
- All requirements must be clear in story file
|
||||||
|
- Optional steps are skipped
|
||||||
|
- Ambiguous stories will halt or skip
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ❌ Agents CANNOT Generate Missing BMAD Sections
|
||||||
|
|
||||||
|
**What this means:**
|
||||||
|
- If story has <12 sections, agent halts
|
||||||
|
- If story has 0 tasks, agent halts
|
||||||
|
- Agent will NOT try to "fix" the story format
|
||||||
|
|
||||||
|
**Why:**
|
||||||
|
- Story format is structural, not implementation
|
||||||
|
- Generating sections requires context agent doesn't have
|
||||||
|
- Gap analysis requires codebase scanning beyond agent scope
|
||||||
|
|
||||||
|
**Implication:**
|
||||||
|
- All stories must be properly formatted BEFORE batch
|
||||||
|
- Run validation: `./scripts/validate-bmad-format.sh`
|
||||||
|
- Regenerate incomplete stories manually
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What Agents CAN Do
|
||||||
|
|
||||||
|
### ✅ Execute Clear, Well-Defined Tasks
|
||||||
|
|
||||||
|
**Works well:**
|
||||||
|
- Stories with 10-30 specific tasks
|
||||||
|
- Clear acceptance criteria
|
||||||
|
- Existing code to modify
|
||||||
|
- Well-defined scope
|
||||||
|
|
||||||
|
### ✅ Make Implementation Decisions
|
||||||
|
|
||||||
|
**Works well:**
|
||||||
|
- Choose between valid approaches
|
||||||
|
- Apply patterns from codebase
|
||||||
|
- Fix bugs based on error messages
|
||||||
|
- Optimize existing code
|
||||||
|
|
||||||
|
### ✅ Run Tests and Verify
|
||||||
|
|
||||||
|
**Works well:**
|
||||||
|
- Execute test suites
|
||||||
|
- Measure coverage
|
||||||
|
- Fix failing tests
|
||||||
|
- Validate implementations
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Pre-Batch Validation Checklist
|
||||||
|
|
||||||
|
**Before running /batch-super-dev, verify ALL selected stories:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Check story files exist
|
||||||
|
for story in $(grep "ready-for-dev" docs/sprint-artifacts/sprint-status.yaml | awk '{print $1}' | sed 's/://'); do
|
||||||
|
[ -f "docs/sprint-artifacts/story-$story.md" ] || echo "❌ Missing: $story"
|
||||||
|
done
|
||||||
|
|
||||||
|
# 2. Check all have 12 BMAD sections
|
||||||
|
for file in docs/sprint-artifacts/story-*.md; do
|
||||||
|
sections=$(grep -c "^## " "$file")
|
||||||
|
if [ "$sections" -lt 12 ]; then
|
||||||
|
echo "❌ Incomplete: $file ($sections/12 sections)"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# 3. Check all have tasks
|
||||||
|
for file in docs/sprint-artifacts/story-*.md; do
|
||||||
|
tasks=$(grep -c "^- \[ \]" "$file")
|
||||||
|
if [ "$tasks" -eq 0 ]; then
|
||||||
|
echo "❌ No tasks: $file"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
**If any checks fail:**
|
||||||
|
1. Regenerate those stories: `/create-story-with-gap-analysis`
|
||||||
|
2. Validate again
|
||||||
|
3. THEN run batch-super-dev
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Error Messages Explained
|
||||||
|
|
||||||
|
### "EARLY BAILOUT: No Tasks Found"
|
||||||
|
|
||||||
|
**What it means:** Story file has 0 unchecked tasks
|
||||||
|
**Is this a bug?** ❌ NO - This is correct validation
|
||||||
|
**What to do:**
|
||||||
|
- If story is skeleton: Regenerate with /create-story-with-gap-analysis
|
||||||
|
- If story is complete: Mark as "done" in sprint-status.yaml
|
||||||
|
- If story needs work: Add tasks to story file
|
||||||
|
|
||||||
|
### "EARLY BAILOUT: Invalid Story Format"
|
||||||
|
|
||||||
|
**What it means:** Story missing required sections (Tasks, AC, etc.)
|
||||||
|
**Is this a bug?** ❌ NO - This is correct validation
|
||||||
|
**What to do:**
|
||||||
|
- Regenerate with /create-story-with-gap-analysis
|
||||||
|
- Do NOT try to manually add sections (skip gap analysis)
|
||||||
|
- Do NOT launch batch with incomplete stories
|
||||||
|
|
||||||
|
### "Story Creation Failed" or "Skipped"
|
||||||
|
|
||||||
|
**What it means:** Agent tried to create story but couldn't
|
||||||
|
**Is this a bug?** ❌ NO - Agents can't create stories
|
||||||
|
**What to do:**
|
||||||
|
- Exit batch-super-dev
|
||||||
|
- Manually run /create-story-with-gap-analysis
|
||||||
|
- Re-run batch after story created
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
### ✅ DO: Generate All Stories Before Batch
|
||||||
|
|
||||||
|
**Workflow:**
|
||||||
|
```
|
||||||
|
1. Plan epic → Identify stories → Create list
|
||||||
|
2. Generate stories: /create-story-with-gap-analysis (1-2 days)
|
||||||
|
3. Validate stories: ./scripts/validate-all-stories.sh
|
||||||
|
4. Execute stories: /batch-super-dev (parallel, fast)
|
||||||
|
```
|
||||||
|
|
||||||
|
### ✅ DO: Use Small Batches for Mixed Complexity
|
||||||
|
|
||||||
|
**Workflow:**
|
||||||
|
```
|
||||||
|
1. Group by complexity (micro, standard, complex)
|
||||||
|
2. Batch micro stories (quick wins)
|
||||||
|
3. Batch standard stories
|
||||||
|
4. Execute complex stories individually
|
||||||
|
```
|
||||||
|
|
||||||
|
### ❌ DON'T: Try to Batch Regenerate
|
||||||
|
|
||||||
|
**Why it fails:**
|
||||||
|
```
|
||||||
|
1. Create 20 skeleton files with just widget lists
|
||||||
|
2. Run /batch-super-dev
|
||||||
|
3. Expect agents to regenerate them
|
||||||
|
→ FAILS: Agents can't invoke /create-story workflow
|
||||||
|
```
|
||||||
|
|
||||||
|
### ❌ DON'T: Mix Skeletons with Proper Stories
|
||||||
|
|
||||||
|
**Why it fails:**
|
||||||
|
```
|
||||||
|
1. 10 proper BMAD stories + 10 skeletons
|
||||||
|
2. Run /batch-super-dev
|
||||||
|
3. Expect batch to handle both
|
||||||
|
→ RESULT: 10 execute, 10 skipped (confusing)
|
||||||
|
```
|
||||||
|
|
||||||
|
### ❌ DON'T: Assume Agents Will "Figure It Out"
|
||||||
|
|
||||||
|
**Why it fails:**
|
||||||
|
```
|
||||||
|
1. Launch batch with unclear stories
|
||||||
|
2. Hope agents will regenerate/fix/create
|
||||||
|
→ RESULT: Agents halt correctly, nothing happens
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
**The Golden Rule:**
|
||||||
|
> **Batch-super-dev is for EXECUTION, not CREATION.**
|
||||||
|
>
|
||||||
|
> Story creation is interactive and requires user input.
|
||||||
|
> Always create/regenerate stories BEFORE batch execution.
|
||||||
|
|
||||||
|
**Remember:**
|
||||||
|
- Agents have limitations (documented above)
|
||||||
|
- These are features, not bugs
|
||||||
|
- Workflows correctly validate and halt
|
||||||
|
- User must prepare stories properly first
|
||||||
|
|
||||||
|
**Success Formula:**
|
||||||
|
```
|
||||||
|
Proper Story Generation (1-2 days manual work)
|
||||||
|
↓
|
||||||
|
Validation (5 minutes automated)
|
||||||
|
↓
|
||||||
|
Batch Execution (4-8 hours parallel autonomous)
|
||||||
|
↓
|
||||||
|
Review & Merge (1-2 hours)
|
||||||
|
```
|
||||||
|
|
||||||
|
Don't skip the preparation steps!
|
||||||
|
|
@ -0,0 +1,347 @@
|
||||||
|
# Batch-Super-Dev Step 2.5 Patch
|
||||||
|
|
||||||
|
**Issue:** Step 2.5 tries to invoke `/create-story-with-gap-analysis` which agents cannot do
|
||||||
|
**Impact:** Skeleton stories get skipped instead of regenerated
|
||||||
|
**Fix:** Explicitly halt batch and tell user to regenerate manually
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Current Code (BROKEN)
|
||||||
|
|
||||||
|
**File:** `instructions.md` lines 82-99
|
||||||
|
|
||||||
|
```xml
|
||||||
|
<ask>Create story file with gap analysis? (yes/no):</ask>
|
||||||
|
|
||||||
|
<check if="response == 'yes'">
|
||||||
|
<output>Creating story {{story_key}} with codebase gap analysis...</output>
|
||||||
|
<action>Invoke workflow: /bmad:bmm:workflows:create-story-with-gap-analysis</action>
|
||||||
|
<action>Parameters: story_key={{story_key}}</action>
|
||||||
|
|
||||||
|
<check if="story creation succeeded">
|
||||||
|
<output>✅ Story {{story_key}} created successfully (12/12 sections)</output>
|
||||||
|
<action>Update file_status_icon to ✅</action>
|
||||||
|
<action>Mark story as validated</action>
|
||||||
|
</check>
|
||||||
|
|
||||||
|
<check if="story creation failed">
|
||||||
|
<output>❌ Story creation failed: {{story_key}}</output>
|
||||||
|
<action>Mark story for removal from selection</action>
|
||||||
|
<action>Add to skipped_stories list with reason: "Creation failed"</action>
|
||||||
|
</check>
|
||||||
|
</check>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
- Line 86: "Invoke workflow: /" doesn't work for agents
|
||||||
|
- Agents can't execute slash commands
|
||||||
|
- This always fails in batch mode
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Recommended Fix (WORKING)
|
||||||
|
|
||||||
|
**Replace lines 82-99 with:**
|
||||||
|
|
||||||
|
```xml
|
||||||
|
<ask>Create story file with gap analysis? (yes/no):</ask>
|
||||||
|
|
||||||
|
<check if="response == 'yes'">
|
||||||
|
<output>
|
||||||
|
⚠️ STORY CREATION REQUIRES MANUAL WORKFLOW EXECUTION
|
||||||
|
|
||||||
|
**Story:** {{story_key}}
|
||||||
|
**Status:** File missing or incomplete
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Agents cannot invoke /create-story-with-gap-analysis workflow autonomously.
|
||||||
|
This workflow requires:
|
||||||
|
- Interactive user prompts
|
||||||
|
- Context-heavy codebase scanning
|
||||||
|
- Gap analysis decision-making
|
||||||
|
|
||||||
|
**Required Action:**
|
||||||
|
|
||||||
|
1. **Exit this batch execution:**
|
||||||
|
- Remaining stories will be skipped
|
||||||
|
- Batch will continue with valid stories only
|
||||||
|
|
||||||
|
2. **Regenerate story manually:**
|
||||||
|
```
|
||||||
|
/create-story-with-gap-analysis
|
||||||
|
```
|
||||||
|
When prompted, provide:
|
||||||
|
- Story key: {{story_key}}
|
||||||
|
- Epic: {epic from parent story}
|
||||||
|
- Scope: {widget list or feature description}
|
||||||
|
|
||||||
|
3. **Validate story format:**
|
||||||
|
```
|
||||||
|
./scripts/validate-bmad-format.sh docs/sprint-artifacts/story-{{story_key}}.md
|
||||||
|
```
|
||||||
|
Must show: "✅ All 12 sections present"
|
||||||
|
|
||||||
|
4. **Re-run batch-super-dev:**
|
||||||
|
- Story will now be properly formatted
|
||||||
|
- Can be executed in next batch run
|
||||||
|
|
||||||
|
**Skipping story {{story_key}} from current batch execution.**
|
||||||
|
</output>
|
||||||
|
|
||||||
|
<action>Mark story for removal from selection</action>
|
||||||
|
<action>Add to skipped_stories list with reason: "Story creation requires manual workflow (agents cannot invoke /create-story)"</action>
|
||||||
|
<action>Add to manual_actions_required list: "Regenerate {{story_key}} with /create-story-with-gap-analysis"</action>
|
||||||
|
</check>
|
||||||
|
|
||||||
|
<check if="response == 'no'">
|
||||||
|
<output>⏭️ Skipping story {{story_key}} (file missing, user declined creation)</output>
|
||||||
|
<action>Mark story for removal from selection</action>
|
||||||
|
<action>Add to skipped_stories list with reason: "User declined story creation"</action>
|
||||||
|
</check>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why This Works:**
|
||||||
|
- ✅ Explicitly states agents can't create stories
|
||||||
|
- ✅ Provides clear step-by-step user actions
|
||||||
|
- ✅ Skips gracefully instead of failing silently
|
||||||
|
- ✅ Tracks manual actions needed
|
||||||
|
- ✅ Sets correct expectations
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Additional Improvements
|
||||||
|
|
||||||
|
### Add Manual Actions Tracking
|
||||||
|
|
||||||
|
**At end of batch execution (Step 5), add:**
|
||||||
|
|
||||||
|
```xml
|
||||||
|
<check if="manual_actions_required is not empty">
|
||||||
|
<output>
|
||||||
|
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
⚠️ MANUAL ACTIONS REQUIRED
|
||||||
|
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
|
||||||
|
**{{manual_actions_required.length}} stories require manual intervention:**
|
||||||
|
|
||||||
|
{{#each manual_actions_required}}
|
||||||
|
{{@index}}. **{{story_key}}**
|
||||||
|
Action: {{action_description}}
|
||||||
|
Command: {{command_to_run}}
|
||||||
|
{{/each}}
|
||||||
|
|
||||||
|
**After completing these actions:**
|
||||||
|
1. Validate all stories: ./scripts/validate-all-stories.sh
|
||||||
|
2. Re-run batch-super-dev for these stories
|
||||||
|
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
</output>
|
||||||
|
</check>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why This Helps:**
|
||||||
|
- User gets clear todo list
|
||||||
|
- Knows exactly what to do next
|
||||||
|
- Can track progress on manual actions
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Validation Script Enhancement
|
||||||
|
|
||||||
|
**Create:** `scripts/validate-all-stories.sh`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
#!/bin/bash
|
||||||
|
# Validate all ready-for-dev stories have proper BMAD format
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
STORIES=$(grep "ready-for-dev" docs/sprint-artifacts/sprint-status.yaml | awk '{print $1}' | sed 's/://')
|
||||||
|
|
||||||
|
echo "=========================================="
|
||||||
|
echo " BMAD Story Format Validation"
|
||||||
|
echo "=========================================="
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
TOTAL=0
|
||||||
|
VALID=0
|
||||||
|
INVALID=0
|
||||||
|
|
||||||
|
for story in $STORIES; do
|
||||||
|
STORY_FILE="docs/sprint-artifacts/story-$story.md"
|
||||||
|
|
||||||
|
if [ ! -f "$STORY_FILE" ]; then
|
||||||
|
echo "❌ $story - FILE MISSING"
|
||||||
|
INVALID=$((INVALID + 1))
|
||||||
|
TOTAL=$((TOTAL + 1))
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check BMAD format
|
||||||
|
./scripts/validate-bmad-format.sh "$STORY_FILE" >/dev/null 2>&1
|
||||||
|
|
||||||
|
if [ $? -eq 0 ]; then
|
||||||
|
echo "✅ $story - Valid BMAD format"
|
||||||
|
VALID=$((VALID + 1))
|
||||||
|
else
|
||||||
|
echo "❌ $story - Invalid format (run validation for details)"
|
||||||
|
INVALID=$((INVALID + 1))
|
||||||
|
fi
|
||||||
|
|
||||||
|
TOTAL=$((TOTAL + 1))
|
||||||
|
done
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=========================================="
|
||||||
|
echo " Summary"
|
||||||
|
echo "=========================================="
|
||||||
|
echo "Total Stories: $TOTAL"
|
||||||
|
echo "Valid: $VALID"
|
||||||
|
echo "Invalid: $INVALID"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ $INVALID -eq 0 ]; then
|
||||||
|
echo "✅ All stories ready for batch execution!"
|
||||||
|
exit 0
|
||||||
|
else
|
||||||
|
echo "❌ $INVALID stories need regeneration"
|
||||||
|
echo ""
|
||||||
|
echo "Run: /create-story-with-gap-analysis"
|
||||||
|
echo "For each invalid story"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why This Helps:**
|
||||||
|
- Quick validation before batch
|
||||||
|
- Prevents wasted time on incomplete stories
|
||||||
|
- Clear pass/fail criteria
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Documentation Update
|
||||||
|
|
||||||
|
**Add to:** `_bmad/bmm/workflows/4-implementation/batch-super-dev/README.md`
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# Batch Super-Dev Workflow
|
||||||
|
|
||||||
|
## Critical Prerequisites
|
||||||
|
|
||||||
|
**BEFORE running batch-super-dev:**
|
||||||
|
|
||||||
|
1. ✅ **All stories must be properly generated**
|
||||||
|
- Run: `/create-story-with-gap-analysis` for each story
|
||||||
|
- Do NOT create skeleton/template files manually
|
||||||
|
- Validation: `./scripts/validate-all-stories.sh`
|
||||||
|
|
||||||
|
2. ✅ **All stories must have 12 BMAD sections**
|
||||||
|
- Business Context, Current State, Acceptance Criteria
|
||||||
|
- Tasks/Subtasks, Technical Requirements, Architecture Compliance
|
||||||
|
- Testing Requirements, Dev Agent Guardrails, Definition of Done
|
||||||
|
- References, Dev Agent Record, Change Log
|
||||||
|
|
||||||
|
3. ✅ **All stories must have tasks**
|
||||||
|
- At least 1 unchecked task (something to implement)
|
||||||
|
- Zero-task stories will be skipped
|
||||||
|
- Validation: `grep -c "^- \[ \]" story-file.md`
|
||||||
|
|
||||||
|
## Common Failure Modes
|
||||||
|
|
||||||
|
### ❌ Attempting Batch Regeneration
|
||||||
|
|
||||||
|
**What you might try:**
|
||||||
|
```
|
||||||
|
1. Create 20 skeleton story files (just headers + widget lists)
|
||||||
|
2. Run /batch-super-dev
|
||||||
|
3. Expect agents to regenerate them
|
||||||
|
```
|
||||||
|
|
||||||
|
**What happens:**
|
||||||
|
- Agents identify stories are incomplete
|
||||||
|
- Agents correctly halt per super-dev-pipeline validation
|
||||||
|
- Stories get skipped (not regenerated)
|
||||||
|
- You waste time
|
||||||
|
|
||||||
|
**Why:**
|
||||||
|
- Agents CANNOT execute /create-story-with-gap-analysis
|
||||||
|
- Agents CANNOT invoke other BMAD workflows
|
||||||
|
- Story generation requires user interaction
|
||||||
|
|
||||||
|
**Solution:**
|
||||||
|
- Generate ALL stories manually FIRST: /create-story-with-gap-analysis
|
||||||
|
- Validate: ./scripts/validate-all-stories.sh
|
||||||
|
- THEN run batch: /batch-super-dev
|
||||||
|
|
||||||
|
### ❌ Mixed Story Quality
|
||||||
|
|
||||||
|
**What you might try:**
|
||||||
|
- Mix 10 proper stories + 10 skeletons
|
||||||
|
- Run batch hoping it "figures it out"
|
||||||
|
|
||||||
|
**What happens:**
|
||||||
|
- 10 proper stories execute successfully
|
||||||
|
- 10 skeletons get skipped
|
||||||
|
- Confusing results
|
||||||
|
|
||||||
|
**Solution:**
|
||||||
|
- Ensure ALL stories have same quality
|
||||||
|
- Validate before batch
|
||||||
|
- Don't mix skeletons with proper stories
|
||||||
|
|
||||||
|
## Success Pattern
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Generate all stories (1-2 days, manual)
|
||||||
|
for story in story-20-13a-{1..5}; do
|
||||||
|
/create-story-with-gap-analysis
|
||||||
|
# Provide story details interactively
|
||||||
|
done
|
||||||
|
|
||||||
|
# 2. Validate (30 seconds, automated)
|
||||||
|
./scripts/validate-all-stories.sh
|
||||||
|
|
||||||
|
# 3. Execute (4-8 hours, parallel autonomous)
|
||||||
|
/batch-super-dev
|
||||||
|
# Select all 5 stories
|
||||||
|
# Choose 2-4 agents parallel
|
||||||
|
|
||||||
|
# 4. Review (1-2 hours)
|
||||||
|
# Review commits, merge to main
|
||||||
|
```
|
||||||
|
|
||||||
|
**Total Time:**
|
||||||
|
- Manual work: 1-2 days (story generation)
|
||||||
|
- Autonomous work: 4-8 hours (batch execution)
|
||||||
|
- Review: 1-2 hours
|
||||||
|
|
||||||
|
**Efficiency:**
|
||||||
|
- Story generation: Cannot be batched (requires user input)
|
||||||
|
- Story execution: Highly parallelizable (4x speedup with 4 agents)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Checklist
|
||||||
|
|
||||||
|
**To apply these improvements:**
|
||||||
|
|
||||||
|
- [ ] Update `batch-super-dev/instructions.md` Step 2.5 (lines 82-99)
|
||||||
|
- [ ] Add `batch-super-dev/AGENT-LIMITATIONS.md` (new file)
|
||||||
|
- [ ] Add `batch-super-dev/BATCH-BEST-PRACTICES.md` (new file)
|
||||||
|
- [ ] Update `batch-super-dev/README.md` with prerequisites
|
||||||
|
- [ ] Create `scripts/validate-all-stories.sh` (new script)
|
||||||
|
- [ ] Add manual actions tracking to Step 5 summary
|
||||||
|
- [ ] Update super-dev-pipeline Step 1.4.5 with agent guidance
|
||||||
|
|
||||||
|
**Testing:**
|
||||||
|
- Try batch with mixed story quality → Should skip skeletons gracefully
|
||||||
|
- Verify error messages are clear
|
||||||
|
- Confirm agents halt correctly (not crash)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Expected Result:**
|
||||||
|
- Users understand limitations upfront
|
||||||
|
- Clear guidance when stories are incomplete
|
||||||
|
- No false expectations about batch regeneration
|
||||||
|
- Better error messages
|
||||||
|
|
@ -4,7 +4,6 @@ header: "Creative Innovation Suite (CIS) Module"
|
||||||
subheader: "No custom configuration required - uses Core settings only"
|
subheader: "No custom configuration required - uses Core settings only"
|
||||||
default_selected: false # This module will not be selected by default for new installations
|
default_selected: false # This module will not be selected by default for new installations
|
||||||
|
|
||||||
|
|
||||||
# Variables from Core Config inserted:
|
# Variables from Core Config inserted:
|
||||||
## user_name
|
## user_name
|
||||||
## communication_language
|
## communication_language
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue