feat: automated learning feedback loop with playbooks (Step 1b + 12)

**New Steps:**
- Step 1b: Load Applicable Playbooks (before gap analysis)
- Step 12: Extract Learnings (after summary)

**How It Works:**
1. Step 1b analyzes story keywords (auto-detect from tasks/title)
2. Searches docs/playbooks/ for matching playbooks
3. Loads applicable playbooks into context
4. Agent uses learnings during implementation
5. Step 12 extracts new patterns from completed work
6. Updates playbooks with learnings
7. Next story benefits from previous work

**Positive Feedback Loop:**
- Story N implements feature → extracts patterns
- Story N+1 loads patterns → implements better
- Gets smarter with every story
- Prevents repeated mistakes

**Self-Contained:**
- No external skill dependencies
- All logic built into workflow steps
- Works with any project (generic)
- Playbook format configurable

**Configuration:**
- learning_feedback section in workflow.yaml
- Keyword auto-detection
- Playbook directory configurable
- Extract/load triggers customizable

**Benefits:**
- Cumulative intelligence across epic
- Prevents repeating same mistakes
- Documents successful patterns
- Builds project-specific knowledge base

Ready for v1.6.0 release with learning feedback!
This commit is contained in:
Jonah Schulte 2026-01-26 09:58:07 -05:00
parent 51324ddb59
commit 14d2cf8f0b
5 changed files with 756 additions and 1 deletions

View File

@ -8,7 +8,7 @@ create_story_workflow: '{project-root}/_bmad/bmm/workflows/4-implementation/crea
# File References
thisStepFile: '{workflow_path}/steps/step-01-init.md'
nextStepFile: '{workflow_path}/steps/step-02-pre-gap-analysis.md'
nextStepFile: '{workflow_path}/steps/step-01b-load-playbooks.md'
# Role
role: null # No agent role yet

View File

@ -0,0 +1,297 @@
# Step 1b: Load Applicable Playbooks (Automated Learning)
**Goal:** Automatically load relevant playbooks and learnings before implementation starts
---
## What This Step Does
Before writing any code, check if previous work has created playbooks/patterns that apply to this story. This creates a **positive feedback loop** where agents get smarter with each story.
---
## Execution
### 1. Analyze Story for Keywords
**Extract topics from story:**
```bash
story_file="{story_file}"
# Extract keywords from:
# - Story title
# - Task descriptions
# - Technical Requirements section
# - Architecture Compliance section
keywords=$(grep -E "^### Task|^## Technical|^## Architecture" "$story_file" | \
grep -oiE "(prisma|stripe|auth|cron|queue|state machine|billing|payment|migration|api|database|email|notification|cache|redis|s3)" | \
sort -u | tr '\n' ',' | sed 's/,$//')
echo "📚 Story keywords: $keywords"
```
**Common keyword categories:**
- **Technology:** Database ORM, payment processor, cache, storage, infrastructure
- **Domain:** Business logic areas (determined by your project)
- **Pattern:** Architecture patterns (state machine, cron, queue, api, etc.)
### 2. Check for Existing Playbooks
**Search playbook directory:**
```bash
playbook_dir="{project-root}/docs/playbooks"
if [ ! -d "$playbook_dir" ]; then
echo "📚 No playbooks directory found - will create after implementation"
playbooks_loaded=0
skip_to_step_2=true
fi
# For each keyword, check if playbook exists
applicable_playbooks=()
for keyword in $(echo "$keywords" | tr ',' ' '); do
playbook_file="${playbook_dir}/${keyword}-playbook.md"
if [ -f "$playbook_file" ]; then
echo "✅ Found playbook: ${keyword}-playbook.md"
applicable_playbooks+=("$playbook_file")
fi
done
echo "📚 Applicable playbooks: ${#applicable_playbooks[@]}"
```
### 3. Load Playbooks (If Found)
**For each applicable playbook:**
```
Read playbook file
Extract sections:
- ## Best Practices
- ## Common Pitfalls
- ## Code Patterns
- ## Lessons Learned
- ## Do's and Don'ts
Store in context for use during implementation
```
**Example:**
```markdown
📚 Loaded: {technology}-playbook.md
Best Practices:
- {Practice learned from previous implementations}
- {Pattern that worked successfully}
- {Standard established across stories}
Common Pitfalls:
- {Mistake that was made and fixed}
- {Issue encountered and resolved}
- {Edge case discovered}
Code Patterns:
- {Reusable pattern with code example}
- {Configuration approach}
- {Testing strategy}
```
### 4. Request Additional Context (If Needed)
**If no playbooks found but keywords detected:**
```
🔍 No existing playbooks for: {keywords}
Checking for project-level guidance...
```
**Use /get-playbook to request specific guidance:**
```bash
# For each keyword without playbook
for keyword in $keywords_without_playbooks; do
echo "📖 Requesting playbook: $keyword"
# Invoke /get-playbook with keyword
# This searches docs/, pulls from project context, creates on-demand playbook
# If guidance found, cache for future use
done
```
### 5. Display Loaded Context
**Output to agent:**
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📚 APPLICABLE LEARNINGS LOADED
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Story: {story_key}
Keywords: {keywords}
Playbooks Loaded: {count}
1. prisma-playbook.md - Database schema best practices
2. state-machine-playbook.md - State transition patterns
3. billing-playbook.md - Payment processing learnings
Key Guidance:
✅ Use 2026 in migration names (not 2025)
✅ Split enum add + use into 2 migrations
✅ Test state transitions exhaustively
✅ Idempotency keys for all charges
✅ Never hardcode payment amounts
These learnings come from previous story implementations.
Use them to avoid repeating past mistakes.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
```
---
## Integration Points
**This step runs:**
- After Step 1 (Init) - story file loaded
- Before Step 2 (Gap Analysis) - context ready
- Before Step 3 (Write Tests) - patterns available
- Before Step 4 (Implement) - guidance active
**Feeds into:**
- Step 2: Gap analysis sees playbook patterns
- Step 3: Test patterns from playbooks
- Step 4: Implementation follows playbook best practices
- Step 7: Code review compares against playbook standards
---
## File Structure
```
docs/
playbooks/
prisma-playbook.md # Prisma best practices
stripe-playbook.md # Stripe integration patterns
auth-playbook.md # Authentication learnings
state-machine-playbook.md # State machine patterns
cron-playbook.md # Cron job patterns
billing-playbook.md # Billing/payment learnings
migration-playbook.md # Database migration rules
```
---
## Playbook Template
```markdown
# {Technology/Domain} Playbook
**Last Updated:** {date}
**Source Stories:** {story-keys that contributed learnings}
## Best Practices
1. {Practice from successful implementations}
2. {Pattern that worked well}
3. {Standard we always follow}
## Common Pitfalls
❌ {Mistake that was made}
✅ {How to avoid it}
❌ {Another mistake}
✅ {Correct approach}
## Code Patterns
**Pattern: {Name}**
```typescript
// Code example from actual implementation
// Story: {story-key}
```
**When to use:** {Context}
**When NOT to use:** {Anti-pattern}
## Lessons Learned
**Story {story-key}:**
- Learned: {Insight}
- Applied: {How we changed approach}
- Result: {Outcome}
## Do's and Don'ts
**DO:**
- {Positive practice}
- {What works}
**DON'T:**
- {What to avoid}
- {Anti-pattern}
## References
- Story implementations: {list of story-keys}
- Architecture docs: {relevant docs}
- External resources: {links}
```
---
## Benefits
**Positive Feedback Loop:**
1. Story 18-1 implements Prisma migration → Learns "use 2026 in names"
2. Story 18-1 complete → Extracts pattern → Saves to prisma-playbook.md
3. Story 18-2 starts → Loads prisma-playbook.md → Applies learning
4. Story 18-2 implements correctly first time (no year mistake)
**Cumulative Intelligence:**
- Epic 1: Creates foundational playbooks
- Epic 2: Builds on Epic 1 playbooks, adds new learnings
- Epic 18: Has 17 epics worth of learnings available
- Gets smarter with every story
**Prevents Repeated Mistakes:**
- Migration year mistakes (happened multiple times)
- Enum transaction limitations (PostgreSQL gotcha)
- Story file naming inconsistencies (just experienced!)
- Missing checkbox updates (just fixed!)
---
## Auto-Load Triggers
**Automatic playbook loading based on story content:**
| Story Contains | Auto-Load Playbooks |
|----------------|---------------------|
| Database ORM keywords | database-orm-playbook.md, migration-playbook.md |
| Payment keywords | payment-playbook.md, billing-playbook.md |
| Auth keywords | auth-playbook.md, security-playbook.md |
| Scheduling keywords | cron-playbook.md, scheduling-playbook.md |
| Queue/async keywords | queue-playbook.md, async-playbook.md |
| State machine keywords | state-machine-playbook.md |
| Email/notification keywords | email-playbook.md, notification-playbook.md |
**Note:** Playbook names are derived from keywords found in your story files.
The system adapts to your project's technology stack.
---
## Next Step
When complete, continue to Step 2 (Pre-Gap Analysis) with loaded playbooks in context.
**Next:** `{workflow_path}/steps/step-02-pre-gap-analysis.md`

View File

@ -10,6 +10,8 @@ thisStepFile: '{workflow_path}/steps/step-11-summary.md'
stateFile: '{state_file}'
storyFile: '{story_file}'
auditTrail: '{audit_trail}'
nextStepFile: '{workflow_path}/steps/step-12-extract-learnings.md'
# Role
role: null

View File

@ -0,0 +1,406 @@
# Step 12: Extract Learnings & Update Playbooks (Automated Feedback Loop)
**Goal:** Extract patterns from completed implementation and save to playbooks for future stories
---
## What This Step Does
After successful implementation, analyze what was built and extract:
- Patterns that worked well
- Mistakes that were made and fixed
- Code examples worth reusing
- Best practices discovered
Save these to playbooks so future stories benefit from this learning.
---
## Execution
### 1. Analyze Implementation
**Review what was built:**
```bash
story_key="{story_key}"
# Get commit for this story
commit_hash=$(git log --oneline --grep="$story_key" -i | head -1 | cut -d' ' -f1)
if [ -z "$commit_hash" ]; then
echo "⚠️ No commit found for $story_key - skipping learning extraction"
exit 0
fi
# Get files changed
files_changed=$(git diff-tree --no-commit-id --name-only -r "$commit_hash")
echo "📊 Analyzing implementation:"
echo " Story: $story_key"
echo " Commit: $commit_hash"
echo " Files: $(echo "$files_changed" | wc -l | tr -d ' ')"
```
### 2. Extract Applicable Keywords
**From story file and code:**
```bash
# Extract from story title, tasks, and file paths
keywords=$(cat "{story_file}" | \
grep -oiE "(prisma|stripe|auth|cron|queue|state.machine|billing|payment|migration|api|database|email|notification|terraform|sqs)" | \
sort -u)
# Also extract from file paths
tech_keywords=$(echo "$files_changed" | \
grep -oiE "(billing|payment|auth|email|queue|cron|migration)" | \
sort -u)
all_keywords=$(echo -e "$keywords\n$tech_keywords" | sort -u | tr '\n' ',' | sed 's/,$//')
echo "🔑 Keywords for playbook matching: $all_keywords"
```
### 3. Check Code Review Findings
**If code review was executed, extract learnings:**
```bash
review_file="docs/sprint-artifacts/review-${story_key}.md"
if [ -f "$review_file" ]; then
echo "📝 Code review exists - extracting learnings"
# Extract issues that were fixed
critical_issues=$(grep -A 3 "CRITICAL" "$review_file" || echo "none")
high_issues=$(grep -A 3 "HIGH" "$review_file" || echo "none")
# These become "Common Pitfalls" in playbooks
fi
```
### 4. Identify Reusable Patterns
**Parse code for patterns worth saving:**
```bash
# Look for:
# - Well-structured functions (with JSDoc, error handling, tests)
# - Reusable utilities
# - Configuration patterns
# - Test fixtures
# Example patterns to extract:
patterns_found=()
# State machine pattern
if echo "$files_changed" | grep -q "state.*machine"; then
patterns_found+=("state-machine")
fi
# Cron pattern
if echo "$files_changed" | grep -q "cron"; then
patterns_found+=("cron-job")
fi
# Migration pattern
if echo "$files_changed" | grep -q "migration"; then
patterns_found+=("database-migration")
fi
echo "🎯 Patterns identified: ${patterns_found[@]}"
```
### 5. Update or Create Playbooks
**For each keyword/pattern:**
```bash
playbook_dir="{project-root}/docs/playbooks"
mkdir -p "$playbook_dir"
for keyword in $(echo "$all_keywords" | tr ',' ' '); do
playbook_file="${playbook_dir}/${keyword}-playbook.md"
if [ -f "$playbook_file" ]; then
echo "📖 Updating existing playbook: ${keyword}-playbook.md"
action="update"
else
echo "📝 Creating new playbook: ${keyword}-playbook.md"
action="create"
fi
# Extract learnings specific to this keyword
# Append to playbook
done
```
### 6. Extract Specific Learnings
**What to extract:**
#### A. Best Practices (from successful implementation)
```
IF story completed with:
- All tests passing
- Code review found zero critical issues
- Clean implementation (no major refactors needed)
THEN extract:
- Code patterns used
- File structure choices
- Testing approaches
- Configuration patterns
```
**Example extraction:**
```markdown
## Best Practices
**Story {story-key} ({story-title}):**
✅ {Successful pattern used}
✅ {Approach that worked well}
✅ {Quality standard met}
✅ {Test coverage achieved}
**Lesson:** {Key insight from implementation}
**Pattern:** See {file-path}:{function-name}()
```
#### B. Common Pitfalls (from mistakes/fixes)
```
IF code review found issues, OR
Implementation required fixes, OR
Tests failed initially
THEN extract:
- What went wrong
- How it was fixed
- How to avoid next time
```
**Example extraction:**
```markdown
## Common Pitfalls
❌ **Story {story-key}: {Mistake description}**
- Problem: {What went wrong}
- Error: {Error message if applicable}
- Root cause: {Why it happened}
- Fix: {How it was resolved}
- Prevention: {How to avoid in future}
❌ **Story {story-key}: {Another mistake}**
- Problem: {Issue encountered}
- Fix: {Solution applied}
- Prevention: {Best practice going forward}
```
#### C. Code Patterns (reusable snippets)
```
IF implementation created reusable code:
- Helper functions
- Type definitions
- Test utilities
- Configuration patterns
THEN extract code snippets with documentation
```
**Example extraction:**
```markdown
## Code Patterns
### Pattern: Idempotency Key Generation
```typescript
// Source: Story 18-1, lib/billing/billing-service.ts
export function generateIdempotencyKey(
agreementId: string,
billingPeriod: string,
chargeType: ChargeType
): string {
return `charge-${agreementId}-${billingPeriod}-${chargeType}`;
}
```
**When to use:** Preventing duplicate charge creation
**Why it works:** Deterministic key based on unique identifiers
**Tests:** See __tests__/lib/billing/billing-service.test.ts:47
```
### 7. Commit Playbook Updates
**If playbooks were created/updated:**
```bash
git add docs/playbooks/*.md
git commit -m "docs: update playbooks from Story ${story_key} learnings
Added/updated:
$(git diff --cached --name-only docs/playbooks/ | sed 's/^/- /')
Learnings from successful implementation of ${story_key}."
```
---
## Output
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ LEARNINGS EXTRACTED
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Playbooks Updated: {count}
- prisma-playbook.md (added migration year validation)
- state-machine-playbook.md (added transition validation pattern)
- billing-playbook.md (added idempotency key pattern)
Patterns Extracted: {count}
- Idempotency key generation
- State machine validation
- Decimal field serialization
Common Pitfalls Documented: {count}
- Migration year sorting issue
- Enum transaction limitation
These learnings will benefit future stories!
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
```
---
## Feedback Loop
```
Story N Implementation
Extract Learnings (Step 12)
Update Playbooks
Commit Playbooks
Story N+1 Starts
Load Playbooks (Step 1b) ← Gets Story N's learnings!
Implement with Knowledge
Repeat
```
**Result:** Each story is smarter than the last.
---
## Skill Integration
**Use existing /extract-patterns skill:**
```bash
# After implementation completes
/extract-patterns story-key={story_key} output-dir=docs/playbooks/
# This extracts:
# - Code patterns
# - Test patterns
# - Configuration patterns
# Saves to playbooks directory
```
**Use /get-playbook for on-demand guidance:**
```bash
# During Step 1b if no playbook exists
/get-playbook topic=prisma-migrations
# Searches project docs for guidance
# Creates playbook if patterns found
```
**Use /get-more-context for comprehensive dives:**
```bash
# For complex stories (COMPLEX complexity level)
/get-more-context topic=stripe-integration
# Loads comprehensive documentation
# More detailed than playbook
# Used for high-stakes implementations
```
---
## Configuration
**In super-dev-pipeline/workflow.yaml:**
```yaml
learning_feedback:
enabled: true # Enable automatic playbook loading/saving
playbook_dir: "{project-root}/docs/playbooks"
# When to extract learnings
extract_triggers:
- on_success: true # Extract after successful completion
- on_code_review: true # Extract from review findings
- on_complex: true # Always extract from COMPLEX stories
# When to load playbooks
load_triggers:
- on_init: true # Load at Step 1b
- keywords_required: 1 # Minimum keywords to trigger
```
---
## Step Sequence Update
**Before (without learning loop):**
1. Init
2. Gap Analysis
3. Write Tests
4. Implement
...
11. Summary
**After (with learning loop):**
1. Init
**1b. Load Playbooks** ← NEW (loads previous learnings)
2. Gap Analysis (informed by playbooks)
3. Write Tests (uses test patterns from playbooks)
4. Implement (follows playbook best practices)
...
11. Summary
**12. Extract Learnings** ← NEW (saves for next story)
---
## Success Criteria
**Step 1b succeeds when:**
- [x] Keywords extracted from story
- [x] Existing playbooks checked
- [x] Applicable playbooks loaded (or noted as missing)
- [x] Context ready for implementation
**Step 12 succeeds when:**
- [x] Learnings extracted from implementation
- [x] Playbooks updated or created
- [x] Patterns documented with code examples
- [x] Playbooks committed to git
---
**Next:** Continue to Step 2 (Pre-Gap Analysis) with loaded playbook context.

View File

@ -78,6 +78,56 @@ smart_batching:
min_batch_size: 3 # Minimum tasks to form a batch
fallback_on_failure: true # Revert to individual if batch fails
# Automated learning feedback loop (NEW v1.6.0)
learning_feedback:
enabled: true # Enable automatic playbook loading/saving
playbook_dir: "{project-root}/docs/playbooks"
# Step 1b: Load existing playbooks before implementation
load_playbooks:
enabled: true
step: "step-01b-load-playbooks.md"
triggers:
- keywords_detected: 1 # Load if any keywords found in story
- complex_story: true # Always load for COMPLEX stories
# Step 12: Extract learnings after implementation
extract_learnings:
enabled: true
step: "step-12-extract-learnings.md"
triggers:
- on_success: true # Extract after successful completion
- on_code_review: true # Extract from review findings
- on_complex: true # Always extract from COMPLEX stories
- skip_if_failed: true # Don't extract from failed implementations
# Keyword extraction
keywords:
auto_detect: true
sources:
- story_title
- task_descriptions
- file_paths
- technical_requirements
common_keywords:
- prisma
- stripe
- auth
- cron
- queue
- state machine
- billing
- payment
- migration
- api
- database
- email
- notification
- terraform
- sqs
- redis
- s3
# Batchable pattern definitions
batchable_patterns:
- pattern: "package_installation"