diff --git a/docs/improvements/party-mode-integration/01-implementation-plan.md b/docs/improvements/party-mode-integration/01-implementation-plan.md new file mode 100644 index 000000000..2ad3898de --- /dev/null +++ b/docs/improvements/party-mode-integration/01-implementation-plan.md @@ -0,0 +1,1086 @@ +# Implementation Plan: Party Mode Integration with Epic Execute + +## Option B: Configurable Party Mode + +**Version**: 1.0.0 +**Status**: Draft +**Author**: BMad Method +**Date**: 2026-01-03 + +--- + +## Related Documents + +| Document | Description | +|----------|-------------| +| [README.md](./README.md) | Overview and quick reference | +| [02-context-management.md](./02-context-management.md) | Deep dive into context isolation and data transfer | +| [03-file-modifications.md](./03-file-modifications.md) | Detailed file change specifications | + +--- + +## Executive Summary + +This plan integrates Party Mode's multi-agent collaboration capabilities into the Epic Execute workflow through configurable CLI flags. Users can enable party phases at specific workflow points to gain diverse agent perspectives during story implementation. + +--- + +## 1. New CLI Flags + +### Shell Script Arguments + +Add the following flags to `scripts/epic-execute.sh`: + +```bash +# Party Mode Integration Flags +--party-kickoff Enable Story Kickoff Party before each story's dev phase +--party-review Replace single-agent review with multi-agent Party Review +--party-failure Enable Failure Analysis Party when stories fail +--party-retro Enable Post-Epic Retrospective Party after all stories +--party-all Enable all party phases (equivalent to all flags above) +--party-agents LIST Override default agents for party phases (comma-separated) +``` + +### Usage Examples + +```bash +# Enable kickoff discussions only +./epic-execute.sh 42 --party-kickoff + +# Full party integration +./epic-execute.sh 42 --party-all + +# Custom review with specific agents +./epic-execute.sh 42 --party-review --party-agents "Winston,Murat,Amelia" + +# Kickoff + Review (most common) +./epic-execute.sh 42 --party-kickoff --party-review + +# With existing flags +./epic-execute.sh 42 --party-all --skip-done --verbose +``` + +--- + +## 2. Configuration File Updates + +### File: `config/default-config.yaml` + +Add new `party` section: + +```yaml +# Party Mode Integration +party: + # Story Kickoff Party - multi-agent discussion before dev phase + kickoff: + enabled: false + agents: + - Winston # Architect - architectural implications + - Amelia # Developer - implementation concerns + - Murat # Test Architect - testing strategy + timeout: 300 # Max seconds for kickoff discussion + output: story # Where to save insights: story | separate | none + + # Party Review - replace single-agent review with multi-agent + review: + enabled: false + agents: + - Winston # Architecture alignment + - Murat # Test coverage, security + - Amelia # Code quality, maintainability + timeout: 600 # Max seconds for review party + consensus_required: false # Require all agents to approve + + # Failure Analysis Party - triggered on story failure + failure_analysis: + enabled: false + agents: + - Winston # Architectural blockers + - Amelia # Implementation issues + - Bob # Process/requirement issues + auto_trigger: true # Auto-trigger on any failure + + # Post-Epic Retrospective Party + retrospective: + enabled: false + agents: + - Mary # Business Analyst - requirements reflection + - Bob # Scrum Master - process reflection + - Winston # Architect - technical reflection + - Amelia # Developer - implementation reflection + generate_handoff: true # Generate rich context handoff document + + # Global party settings + settings: + # Communication language for all party discussions + language: "{{communication_language}}" + + # Enable TTS for party responses + tts_enabled: false + + # Max agents per party phase + max_agents: 4 + + # Party output format: markdown | yaml | json + output_format: markdown +``` + +--- + +## 3. Workflow Integration Points + +### Enhanced Epic Execute Flow Diagram + +``` +┌─────────────────────────────────────────────────────────────────────────┐ +│ ENHANCED EPIC EXECUTE FLOW │ +├─────────────────────────────────────────────────────────────────────────┤ +│ │ +│ For each story: │ +│ ┌─────────────────────────────────────────────────────────────────┐ │ +│ │ │ │ +│ │ ┌──────────────────┐ │ │ +│ │ │ PARTY: Kickoff │ ← --party-kickoff │ │ +│ │ │ (Optional) │ Winston + Amelia + Murat │ │ +│ │ │ │ Output: Implementation strategy │ │ +│ │ └────────┬─────────┘ Saved to: Story file or separate doc │ │ +│ │ │ │ │ +│ │ ▼ │ │ +│ │ ┌──────────────────┐ │ │ +│ │ │ Phase 1: Dev │ Context A (Isolated) │ │ +│ │ │ (Standard) │ Includes kickoff insights if generated │ │ +│ │ └────────┬─────────┘ │ │ +│ │ │ │ │ +│ │ ▼ success ▼ failure │ │ +│ │ ┌──────────────────┐ ┌──────────────────┐ │ │ +│ │ │ Phase 2: Review │ │ PARTY: Failure │ │ │ +│ │ │ │ │ Analysis │ ← --party-fail│ │ +│ │ │ --party-review? │ │ (Optional) │ │ │ +│ │ │ ┌──────────────┐ │ └────────┬─────────┘ │ │ +│ │ │ │ YES: Party │ │ │ │ │ +│ │ │ │ Review │ │ ▼ │ │ +│ │ │ │ Winston + │ │ ┌──────────────────┐ │ │ +│ │ │ │ Murat + │ │ │ Retry or Skip │ │ │ +│ │ │ │ Amelia │ │ └──────────────────┘ │ │ +│ │ │ └──────────────┘ │ │ │ +│ │ │ ┌──────────────┐ │ │ │ +│ │ │ │ NO: Standard │ │ │ │ +│ │ │ │ Single-agent │ │ │ │ +│ │ │ │ Review │ │ │ │ +│ │ │ └──────────────┘ │ │ │ +│ │ └────────┬─────────┘ │ │ +│ │ │ │ │ +│ │ ▼ │ │ +│ │ ┌──────────────────┐ │ │ +│ │ │ Phase 3: Commit │ Shell orchestration │ │ +│ │ └──────────────────┘ │ │ +│ │ │ │ +│ └─────────────────────────────────────────────────────────────────┘ │ +│ │ +│ After all stories: │ +│ ┌─────────────────────────────────────────────────────────────────┐ │ +│ │ │ │ +│ │ ┌──────────────────┐ ┌──────────────────┐ │ │ +│ │ │ Phase 4: UAT │ │ PARTY: Retro │ ← --party-retro │ │ +│ │ │ Generation │ │ (Optional) │ │ │ +│ │ │ (Standard) │ │ Mary + Bob + │ │ │ +│ │ │ │ │ Winston + Amelia │ │ │ +│ │ └──────────────────┘ │ │ │ │ +│ │ │ Output: │ │ │ +│ │ │ - Retro insights │ │ │ +│ │ │ - Context handoff│ │ │ +│ │ │ - Patterns doc │ │ │ +│ │ └──────────────────┘ │ │ +│ │ │ │ +│ └─────────────────────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## 4. New Step Files + +### 4.1 Story Kickoff Party + +**File**: `steps/step-01b-party-kickoff.md` + +```markdown +# Step 1b: Story Kickoff Party + +## Purpose + +Bring together 2-3 agents for a focused discussion before implementation begins. +Surface architectural concerns, implementation challenges, and testing strategies. + +## Agents + +Default: Winston (Architect), Amelia (Developer), Murat (Test Architect) +Override: Via --party-agents flag or config + +## Input Context + +- Story specification (acceptance criteria, technical context) +- Architecture document +- Related stories in the epic + +## Discussion Topics + +1. **Architectural Implications** + - Winston: How does this fit the overall architecture? + - What integration points exist? + - Are there scalability concerns? + +2. **Implementation Approach** + - Amelia: What patterns should we follow? + - Are there existing utilities to leverage? + - What are the tricky parts? + +3. **Testing Strategy** + - Murat: What test types are needed? + - What edge cases should we cover? + - Are there test fixtures we need? + +## Output Format + +```yaml +## Story Kickoff Insights + +**Discussion Date**: {{date}} +**Participants**: {{agent_list}} + +### Architectural Notes +[Winston's key points] + +### Implementation Strategy +[Amelia's recommendations] + +### Testing Approach +[Murat's test strategy] + +### Identified Risks +- [Risk 1] +- [Risk 2] + +### Decisions Made +- [Decision 1 with rationale] +``` + +## Success Criteria (Analysis Party) + +- All agents contributed perspective +- Clear implementation direction established +- Risks identified and documented +- Ready for dev phase to proceed +``` + +--- + +### 4.2 Party Review + +**File**: `steps/step-03b-party-review.md` + +```markdown +# Step 3b: Party Review (Multi-Agent Code Review) + +## Purpose + +Replace single-agent code review with multi-agent collaborative review. +Each agent focuses on their domain expertise for thorough coverage. + +## Agents + +Default: Winston (Architect), Murat (Test Architect), Amelia (Developer) +Override: Via --party-agents flag or config + +## Agent Focus Areas + +### Winston (Architecture) +- Pattern adherence +- Scalability implications +- API design consistency +- Component boundaries +- Integration concerns + +### Murat (Quality) +- Test coverage completeness +- Test quality and meaningfulness +- Security vulnerabilities +- Edge case handling +- CI/CD implications + +### Amelia (Implementation) +- Code quality and readability +- Error handling +- Performance considerations +- Documentation quality +- Maintainability + +## Review Protocol + +1. **Independent Analysis** (per agent) + - Each agent reviews from their perspective + - Categorizes findings by severity (HIGH/MEDIUM/LOW) + +2. **Cross-Discussion** + - Agents discuss findings + - Resolve conflicting opinions + - Prioritize issues collectively + +3. **Consensus Building** + - Agree on which issues block approval + - Agree on fix priorities + - Generate unified review record + +## Output Format + +```yaml +## Party Review Record + +**Review Date**: {{date}} +**Reviewers**: {{agent_list}} + +### Agent Findings + +#### 🏗️ Winston (Architecture) +| # | Finding | Severity | Recommendation | +|---|---------|----------|----------------| + +#### 🧪 Murat (Quality) +| # | Finding | Severity | Recommendation | +|---|---------|----------|----------------| + +#### 💻 Amelia (Implementation) +| # | Finding | Severity | Recommendation | +|---|---------|----------|----------------| + +### Cross-Discussion Notes +[Key discussion points and resolutions] + +### Consensus Decision +- **Status**: Approved | Approved with Fixes | Rejected +- **Blocking Issues**: [List if any] +- **Required Fixes**: [Prioritized list] + +### Fixes Applied +[List of changes made during review] +``` + +## Issue Fix Policy + +Same as standard review: +- HIGH: Always fix +- MEDIUM: Fix if total > 5 +- LOW: Document only + +## Success Criteria (Review Party) + +- All agents completed their review focus +- Issues categorized and prioritized +- Consensus reached on approval status +- Required fixes applied and verified +``` + +--- + +### 4.3 Failure Analysis Party + +**File**: `steps/step-02b-party-failure.md` + +```markdown +# Step 2b: Failure Analysis Party + +## Purpose + +When a story fails (dev blocked or review failed), convene agents to: +- Diagnose root cause +- Propose remediation +- Identify process improvements + +## Trigger Conditions + +- Dev phase outputs: `IMPLEMENTATION BLOCKED` +- Review phase outputs: `REVIEW FAILED` +- Test failures after max retries + +## Agents + +Default: Winston (Architect), Amelia (Developer), Bob (Scrum Master) +Override: Via --party-agents flag + +## Discussion Protocol + +1. **Failure Context Sharing** + - Present the failure message/log + - Share relevant code context + - Show what was attempted + +2. **Root Cause Analysis** + - Winston: Is this an architectural issue? + - Amelia: Is this a technical implementation issue? + - Bob: Is this a requirements/process issue? + +3. **Remediation Planning** + - What needs to change? + - Who/what is best positioned to fix it? + - Are there blocking dependencies? + +4. **Process Improvement** + - Could this have been caught earlier? + - What should we do differently next time? + +## Output Format + +```yaml +## Failure Analysis Record + +**Analysis Date**: {{date}} +**Story**: {{story_id}} +**Failure Type**: Dev Blocked | Review Failed | Test Failure +**Analysts**: {{agent_list}} + +### Failure Summary +[What happened] + +### Root Cause Analysis + +#### Winston's Assessment +[Architectural perspective] + +#### Amelia's Assessment +[Implementation perspective] + +#### Bob's Assessment +[Process/requirements perspective] + +### Agreed Root Cause +[Consensus diagnosis] + +### Remediation Plan +1. [Step 1] +2. [Step 2] +3. [Step 3] + +### Blocking Dependencies +- [Dependency 1] + +### Process Improvements +- [Improvement for future] + +### Recommendation +- **Action**: Retry | Skip | Escalate to Human +- **Rationale**: [Why this action] +``` + +## Success Criteria (Failure Analysis) + +- Root cause identified +- Remediation path clear +- Actionable next step determined +``` + +--- + +### 4.4 Post-Epic Retrospective Party + +**File**: `steps/step-05b-party-retro.md` + +```markdown +# Step 5b: Post-Epic Retrospective Party + +## Purpose + +After all stories complete, conduct a multi-agent retrospective to: +- Reflect on what worked and what didn't +- Capture patterns and decisions for future reference +- Generate rich context handoff for epic-chain workflows + +## Agents + +Default: Mary (Analyst), Bob (Scrum Master), Winston (Architect), Amelia (Developer) +Override: Via --party-agents flag + +## Input Context + +- All completed story files (with Dev Agent Records and Code Review Records) +- Epic specification +- Execution log/summary + +## Discussion Topics + +### 1. What Went Well +- Mary: Were requirements clear? Did implementation match intent? +- Bob: How was the sprint flow? Were stories well-sized? +- Winston: Did architecture hold up? Good technical decisions? +- Amelia: Code quality? Patterns established? Developer experience? + +### 2. What Could Improve +- Mary: Requirement gaps or ambiguities discovered? +- Bob: Process bottlenecks or inefficiencies? +- Winston: Architectural debt introduced? Future concerns? +- Amelia: Implementation struggles? Missing tooling? + +### 3. Patterns Established +- What coding patterns emerged? +- What testing patterns were effective? +- What documentation patterns helped? + +### 4. Knowledge Transfer +- What should the next epic's team know? +- What gotchas did we discover? +- What shortcuts are now available? + +## Output Format + +```yaml +## Epic {{epic_id}} Retrospective + +**Date**: {{date}} +**Participants**: {{agent_list}} + +### What Went Well +#### Requirements & Business (Mary) +[Points] + +#### Process & Flow (Bob) +[Points] + +#### Architecture & Design (Winston) +[Points] + +#### Implementation & Code (Amelia) +[Points] + +### Areas for Improvement +#### Requirements & Business (Mary) +[Points] + +#### Process & Flow (Bob) +[Points] + +#### Architecture & Design (Winston) +[Points] + +#### Implementation & Code (Amelia) +[Points] + +### Patterns Established +| Pattern | Description | Files | +|---------|-------------|-------| +| [Name] | [What] | [Where] | + +### Key Decisions Log +| Decision | Rationale | Impact | +|----------|-----------|--------| +| [What] | [Why] | [Effect] | + +### Gotchas & Lessons Learned +1. [Gotcha]: [What to watch for] +2. [Lesson]: [What we learned] + +### Context Handoff for Next Epic +[Rich summary for epic-chain context transfer] +``` + +## Integration with Epic-Chain + +When `generate_handoff: true`, output is also saved to: +`docs/handoffs/epic-{{epic_id}}-handoff.md` + +This file is automatically loaded by epic-chain for the next epic. + +## Success Criteria (Retrospective) + +- All agents contributed reflections +- Actionable improvements identified +- Patterns documented for reuse +- Context handoff generated (if configured) +``` + +--- + +## 5. Shell Script Modifications + +### File: `scripts/epic-execute.sh` + +#### 5.1 New Variables + +```bash +# Party Mode Flags +PARTY_KICKOFF=false +PARTY_REVIEW=false +PARTY_FAILURE=false +PARTY_RETRO=false +PARTY_AGENTS="" + +# Party Mode Step Files +PARTY_KICKOFF_STEP="$BMAD_DIR/workflows/4-implementation/epic-execute/steps/step-01b-party-kickoff.md" +PARTY_REVIEW_STEP="$BMAD_DIR/workflows/4-implementation/epic-execute/steps/step-03b-party-review.md" +PARTY_FAILURE_STEP="$BMAD_DIR/workflows/4-implementation/epic-execute/steps/step-02b-party-failure.md" +PARTY_RETRO_STEP="$BMAD_DIR/workflows/4-implementation/epic-execute/steps/step-05b-party-retro.md" +``` + +### 5.2 Argument Parsing Additions + +```bash +while [[ $# -gt 0 ]]; do + case $1 in + # ... existing flags ... + + --party-kickoff) + PARTY_KICKOFF=true + shift + ;; + --party-review) + PARTY_REVIEW=true + shift + ;; + --party-failure) + PARTY_FAILURE=true + shift + ;; + --party-retro) + PARTY_RETRO=true + shift + ;; + --party-all) + PARTY_KICKOFF=true + PARTY_REVIEW=true + PARTY_FAILURE=true + PARTY_RETRO=true + shift + ;; + --party-agents) + PARTY_AGENTS="$2" + shift 2 + ;; + # ... rest of cases ... + esac +done +``` + +#### 5.3 New Functions + +```bash +# ============================================================================= +# Party Mode Functions +# ============================================================================= + +execute_party_kickoff() { + local story_file="$1" + local story_id=$(basename "$story_file" .md) + + log ">>> PARTY KICKOFF: $story_id" + + local story_contents=$(cat "$story_file") + local agents="${PARTY_AGENTS:-Winston,Amelia,Murat}" + + local kickoff_prompt="You are orchestrating a Story Kickoff Party for BMAD. + +## Participating Agents +$agents + +## Story to Discuss + +$story_contents + + +## Your Task +Facilitate a focused discussion between the agents about this story. +Each agent should contribute from their expertise: +- Winston (Architect): Architectural implications, integration points +- Amelia (Developer): Implementation approach, patterns to follow +- Murat (Test Architect): Testing strategy, edge cases + +## Output +Generate a Story Kickoff Insights section to append to the story file. +Format as markdown with clear sections for each agent's input. + +When complete, output: KICKOFF COMPLETE: $story_id" + + if [ "$DRY_RUN" = true ]; then + echo "[DRY RUN] Would execute party kickoff for $story_id" + return 0 + fi + + local result + result=$(claude --dangerously-skip-permissions -p "$kickoff_prompt" 2>&1) || true + + echo "$result" >> "$LOG_FILE" + + if echo "$result" | grep -q "KICKOFF COMPLETE"; then + log_success "Party kickoff complete: $story_id" + return 0 + else + log_warn "Party kickoff may not have completed cleanly" + return 0 # Non-blocking - continue to dev phase + fi +} + +execute_party_review() { + local story_file="$1" + local story_id=$(basename "$story_file" .md) + + log ">>> PARTY REVIEW: $story_id (multi-agent)" + + local story_contents=$(cat "$story_file") + local agents="${PARTY_AGENTS:-Winston,Murat,Amelia}" + + local review_prompt="You are orchestrating a Party Code Review for BMAD. + +## Participating Agents +$agents + +## Story Being Reviewed + +$story_contents + + +## Review Focus Areas +- Winston (Architecture): Pattern adherence, scalability, API design +- Murat (Quality): Test coverage, security, edge cases +- Amelia (Implementation): Code quality, readability, maintainability + +## Your Task +1. Run: git diff --staged +2. Each agent reviews from their perspective +3. Categorize issues by severity (HIGH/MEDIUM/LOW) +4. Facilitate cross-discussion to reach consensus +5. Apply fix policy: HIGH always, MEDIUM if >5 total, LOW document only +6. Generate Party Review Record + +## Completion +If PASSED: Update story Status to Done, output: PARTY REVIEW PASSED: $story_id +If FAILED: Update story Status to Blocked, output: PARTY REVIEW FAILED: $story_id - [reason]" + + if [ "$DRY_RUN" = true ]; then + echo "[DRY RUN] Would execute party review for $story_id" + return 0 + fi + + local result + result=$(claude --dangerously-skip-permissions -p "$review_prompt" 2>&1) || true + + echo "$result" >> "$LOG_FILE" + + if echo "$result" | grep -q "PARTY REVIEW PASSED"; then + log_success "Party review passed: $story_id" + return 0 + elif echo "$result" | grep -q "PARTY REVIEW FAILED"; then + log_error "Party review failed: $story_id" + return 1 + else + log_warn "Party review did not complete cleanly" + return 1 + fi +} + +execute_party_failure_analysis() { + local story_file="$1" + local story_id=$(basename "$story_file" .md) + local failure_type="$2" # "dev" or "review" + + log ">>> PARTY FAILURE ANALYSIS: $story_id" + + local story_contents=$(cat "$story_file") + local agents="${PARTY_AGENTS:-Winston,Amelia,Bob}" + + local failure_prompt="You are orchestrating a Failure Analysis Party for BMAD. + +## Participating Agents +$agents + +## Failed Story + +$story_contents + + +## Failure Type +$failure_type phase failed + +## Your Task +1. Present failure context to agents +2. Facilitate root cause analysis: + - Winston: Architectural issues? + - Amelia: Implementation issues? + - Bob: Requirements/process issues? +3. Build remediation plan +4. Recommend action: Retry | Skip | Escalate + +## Output +Generate Failure Analysis Record. +Output: ANALYSIS COMPLETE: $story_id - [Retry|Skip|Escalate]" + + if [ "$DRY_RUN" = true ]; then + echo "[DRY RUN] Would execute failure analysis for $story_id" + return 0 + fi + + local result + result=$(claude --dangerously-skip-permissions -p "$failure_prompt" 2>&1) || true + + echo "$result" >> "$LOG_FILE" + log_success "Failure analysis complete: $story_id" +} + +execute_party_retro() { + log ">>> PARTY RETROSPECTIVE: Epic $EPIC_ID" + + local epic_contents=$(cat "$EPIC_FILE") + local agents="${PARTY_AGENTS:-Mary,Bob,Winston,Amelia}" + + local all_stories="" + for story_file in "${STORIES[@]}"; do + local story_id=$(basename "$story_file" .md) + all_stories+=" + +$(cat "$story_file") + +" + done + + local retro_prompt="You are orchestrating a Post-Epic Retrospective Party for BMAD. + +## Participating Agents +$agents + +## Epic Completed + +$epic_contents + + +## Completed Stories +$all_stories + +## Your Task +Facilitate retrospective discussion: + +1. What Went Well (each agent's perspective) +2. Areas for Improvement (each agent's perspective) +3. Patterns Established (document for reuse) +4. Key Decisions Log +5. Gotchas & Lessons Learned +6. Context Handoff for Next Epic + +## Output +Generate retrospective document at: docs/sprints/epic-${EPIC_ID}-retro.md +Generate handoff document at: docs/handoffs/epic-${EPIC_ID}-handoff.md + +When complete, output: RETRO COMPLETE: Epic $EPIC_ID" + + if [ "$DRY_RUN" = true ]; then + echo "[DRY RUN] Would execute party retrospective" + return 0 + fi + + local result + result=$(claude --dangerously-skip-permissions -p "$retro_prompt" 2>&1) || true + + echo "$result" >> "$LOG_FILE" + + if echo "$result" | grep -q "RETRO COMPLETE"; then + log_success "Party retrospective complete" + else + log_warn "Retrospective may not have completed cleanly" + fi +} +``` + +#### 5.4 Main Loop Modifications + +```bash +# ============================================================================= +# Main Execution Loop (Modified) +# ============================================================================= + +for story_file in "${STORIES[@]}"; do + story_id=$(basename "$story_file" .md) + + # ... existing skip logic ... + + echo "" + log "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + log "Story: $story_id" + log "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + + # PARTY KICKOFF (Optional - Context 0) + if [ "$PARTY_KICKOFF" = true ]; then + execute_party_kickoff "$story_file" + fi + + # DEV PHASE (Context 1) + if ! execute_dev_phase "$story_file"; then + log_error "Dev phase failed for $story_id" + + # PARTY FAILURE ANALYSIS (Optional) + if [ "$PARTY_FAILURE" = true ]; then + execute_party_failure_analysis "$story_file" "dev" + fi + + ((FAILED++)) + continue + fi + + # REVIEW PHASE (Context 2 - Fresh) + if [ "$SKIP_REVIEW" = false ]; then + if [ "$PARTY_REVIEW" = true ]; then + # Multi-agent party review + if ! execute_party_review "$story_file"; then + log_error "Party review failed for $story_id" + + if [ "$PARTY_FAILURE" = true ]; then + execute_party_failure_analysis "$story_file" "review" + fi + + ((FAILED++)) + continue + fi + else + # Standard single-agent review + if ! execute_review_phase "$story_file"; then + log_error "Review phase failed for $story_id" + ((FAILED++)) + continue + fi + fi + fi + + # COMMIT + commit_story "$story_id" + + ((COMPLETED++)) + log_success "Story complete: $story_id ($COMPLETED/${#STORIES[@]})" +done + +# ============================================================================= +# Post-Epic Activities +# ============================================================================= + +# UAT Generation (Standard) +generate_uat + +# PARTY RETROSPECTIVE (Optional) +if [ "$PARTY_RETRO" = true ]; then + execute_party_retro +fi +``` + +--- + +## 6. File Structure After Implementation + +``` +src/modules/bmm/workflows/4-implementation/epic-execute/ +├── workflow.md # Updated with party mode references +├── config/ +│ └── default-config.yaml # Updated with party section +├── steps/ +│ ├── step-01-init.md # Existing +│ ├── step-01b-party-kickoff.md # NEW: Story Kickoff Party +│ ├── step-02-dev-story.md # Existing +│ ├── step-02b-party-failure.md # NEW: Failure Analysis Party +│ ├── step-03-code-review.md # Existing (used when party-review disabled) +│ ├── step-03b-party-review.md # NEW: Multi-Agent Party Review +│ ├── step-04-generate-uat.md # Existing +│ ├── step-05-summary.md # Existing +│ └── step-05b-party-retro.md # NEW: Post-Epic Retrospective Party +└── prompts/ + ├── kickoff-party.md # Full prompt template + ├── review-party.md # Full prompt template + ├── failure-party.md # Full prompt template + └── retro-party.md # Full prompt template + +scripts/ +└── epic-execute.sh # Updated with party functions +``` + +--- + +## 7. Implementation Phases + +### Phase 1: Foundation (Priority: High) +1. Add CLI flag parsing to `epic-execute.sh` +2. Add party section to `default-config.yaml` +3. Create `step-01b-party-kickoff.md` +4. Implement `execute_party_kickoff()` function +5. Wire kickoff into main loop + +### Phase 2: Core Integration (Priority: High) +1. Create `step-03b-party-review.md` +2. Implement `execute_party_review()` function +3. Add conditional logic for party vs standard review +4. Test party review with sample epic + +### Phase 3: Error Handling (Priority: Medium) +1. Create `step-02b-party-failure.md` +2. Implement `execute_party_failure_analysis()` function +3. Wire failure analysis into dev/review failure paths + +### Phase 4: Retrospective & Handoff (Priority: Medium) +1. Create `step-05b-party-retro.md` +2. Implement `execute_party_retro()` function +3. Integrate with epic-chain context handoff system +4. Create handoff document templates + +### Phase 5: Polish (Priority: Low) +1. Add `--party-agents` override support +2. Add TTS integration for party phases +3. Create comprehensive documentation +4. Add metrics tracking for party phases + +--- + +## 8. Testing Strategy + +### Unit Tests +- Flag parsing works correctly +- Config loading includes party section +- Agent list parsing handles various formats + +### Integration Tests +```bash +# Test kickoff only +./epic-execute.sh test-epic --party-kickoff --dry-run + +# Test full party mode +./epic-execute.sh test-epic --party-all --dry-run + +# Test custom agents +./epic-execute.sh test-epic --party-review --party-agents "Winston,Murat" --dry-run +``` + +### Manual Validation +- Run with real epic to validate agent interactions +- Verify output document quality +- Confirm context isolation still works with party phases + +--- + +## 9. Rollback Plan + +Party mode is additive and opt-in: +- All flags default to `false` +- Standard workflow unchanged when flags not used +- Can disable individual party phases independently +- No changes to existing step files + +--- + +## 10. Success Metrics + +| Metric | Target | Measurement | +|--------|--------|-------------| +| Issue detection rate | +25% | Compare issues found in party review vs standard | +| Architectural issues caught early | +40% | Track issues surfaced in kickoff | +| Context handoff quality | Subjective | Developer satisfaction with handoff docs | +| Failure remediation time | -30% | Time from failure to successful retry | + +--- + +## 11. Future Enhancements + +1. **Complexity-Based Auto-Trigger**: Automatically enable party phases for high-complexity stories +2. **Agent Recommendation Engine**: Suggest optimal agents based on story content +3. **Party Analytics**: Track which agent combinations produce best results +4. **Async Party Mode**: Run party phases in background while other work continues +5. **Human-in-Party**: Allow human to join party discussions at key decision points diff --git a/epic-1-uat-sample.md b/epic-1-uat-sample.md new file mode 100644 index 000000000..2e2a21594 --- /dev/null +++ b/epic-1-uat-sample.md @@ -0,0 +1,526 @@ +# User Acceptance Testing: Epic 1 - Foundation, CLI & Deployment Infrastructure + +**Version:** 1.0 +**Date:** January 2, 2026 +**Epic:** Foundation, CLI & Deployment Infrastructure +**Status:** Ready for Testing + +--- + +## 1. Overview + +### What Was Built + +Epic 1 delivers the foundation of the Heimdall Customer Management system. After completing this epic, administrators can: + +- **Initialize a new Heimdall project** using a simple command-line tool +- **Configure connections** to Supabase (your database) and Resend (your email service) +- **Set up the database** with all necessary tables for job processing +- **Run a worker process** that continuously processes background jobs +- **Deploy to Railway** (a cloud hosting platform) for production use +- **Send test emails** to verify your email configuration is working + +In plain terms: This is the "plumbing" that makes everything else possible. Once Epic 1 is working, you have a running system ready to handle customer management workflows. + +--- + +## 2. Prerequisites + +### Test Environment Requirements + +Before starting UAT, ensure you have: + +| Requirement | Details | How to Verify | +|-------------|---------|---------------| +| **Node.js 20.x** | JavaScript runtime | Run `node --version` - should show v20.x.x | +| **npm** | Package manager | Run `npm --version` - should show 9.x or higher | +| **Git** | Version control | Run `git --version` | +| **Terminal access** | Command line interface | macOS Terminal, Windows PowerShell, or Linux terminal | + +### Required Accounts + +| Service | Purpose | Sign-up URL | +|---------|---------|-------------| +| **Supabase** | Database hosting | (free tier available) | +| **Resend** | Email delivery | (free tier: 100 emails/day) | +| **Railway** (optional) | Cloud deployment | (for production testing) | + +### Credentials You Will Need + +Before testing, gather these from your service dashboards: + +1. **From Supabase Dashboard:** + - Project URL (e.g., `https://xxxxx.supabase.co`) + - Anon/Public Key (starts with `eyJ...`) + - Database Connection String (from Settings > Database > Connection string > URI) + +2. **From Resend Dashboard:** + - API Key (starts with `re_...`) + - A verified sending domain (or use their default for testing) + +3. **Your Information:** + - Admin email address (for receiving notifications and test emails) + - Workspace name (a label for your installation) + +--- + +## 3. Test Scenarios + +### Scenario 1: Project Initialization + +**Goal:** Verify that you can create a new Heimdall project with the correct structure. + +**Steps:** + +1. Open your terminal application +2. Navigate to a folder where you want to create the project: + ``` + cd ~/Documents + ``` +3. Clone and set up the Heimdall project: + ``` + git clone heimdall-test + cd heimdall-test + npm install + npm run build + ``` +4. Verify the CLI is available: + ``` + npx heimdall --version + ``` + +**Expected Results:** + +- [ ] `npm install` completes without errors +- [ ] `npm run build` shows "5 packages built successfully" or similar +- [ ] `npx heimdall --version` displays a version number (e.g., `1.0.0`) +- [ ] `npx heimdall --help` shows available commands including `config`, `db`, `start`, `test-send`, `test-queue` + +**Notes for Tester:** +_Record any error messages or unexpected behavior here:_ + +--- + +### Scenario 2: Configuration Setup + +**Goal:** Create and validate your configuration file with Supabase and Resend credentials. + +**Steps:** + +1. Generate a configuration template: + ``` + npx heimdall config init + ``` +2. Open the created file `heimdall.config.yaml` in a text editor +3. Replace the placeholder values with your real credentials: + - Under `workspace:` - enter your workspace name and admin email + - Under `supabase:` - enter your Supabase URL, anon key, and database URL + - Under `resend:` - enter your Resend API key +4. Save the file +5. Validate your configuration: + ``` + npx heimdall config validate + ``` + +**Expected Results:** + +- [ ] `config init` creates a file named `heimdall.config.yaml` +- [ ] The file contains sections for `workspace`, `supabase`, `resend`, and `ai` (optional) +- [ ] `config validate` shows your workspace name and admin email (with secrets partially hidden) +- [ ] Validation shows "Configuration is valid" or similar success message +- [ ] No error messages appear about missing or invalid fields + +**Notes for Tester:** +_If validation fails, record the error message:_ + +--- + +### Scenario 3: Database Migration + +**Goal:** Set up the required database tables in your Supabase instance. + +**Prerequisites:** Scenario 2 must be completed successfully. + +**Steps:** + +1. Run the database migration: + ``` + npx heimdall db migrate + ``` +2. Check the database status: + ``` + npx heimdall db status + ``` + +**Expected Results:** + +- [ ] `db migrate` completes with a success message (e.g., "pg-boss initialized successfully") +- [ ] `db status` shows: + - Database connected: Yes + - pg-boss schema: Exists + - Tables: job, schedule, subscription, version (or similar) +- [ ] Running `db migrate` a second time does NOT cause errors (idempotent) + +**Verification in Supabase Dashboard:** +1. Log into your Supabase project +2. Go to the Table Editor +3. Look for a schema named `pgboss` +4. Verify tables exist: `job`, `schedule`, `subscription`, `version` + +- [ ] pg-boss tables visible in Supabase Dashboard + +**Notes for Tester:** +_Record database status output:_ + +--- + +### Scenario 4: Connection Validation (Detailed) + +**Goal:** Verify both Supabase API and direct database connections work correctly. + +**Prerequisites:** Scenario 3 must be completed successfully. + +**Steps:** + +1. Run the full validation: + ``` + npx heimdall config validate + ``` + +**Expected Results:** + +- [ ] Message: "Supabase API connected" with response time in milliseconds +- [ ] Message: "Operational DB connected" with response time in milliseconds +- [ ] Message: "Resend API connected" (if Resend key is configured) +- [ ] All connections show as successful (green checkmarks or similar) + +**Notes for Tester:** +_Record connection times and any warnings:_ + +--- + +### Scenario 5: Worker Process Startup + +**Goal:** Start the background worker that processes jobs. + +**Prerequisites:** Scenario 3 must be completed successfully. + +**Steps:** + +1. Start the worker process: + ``` + npx heimdall start + ``` +2. Observe the output for approximately 30 seconds +3. Press `Ctrl+C` to stop the worker + +**Expected Results:** + +- [ ] Message appears: "Heimdall worker started, polling for jobs..." or similar +- [ ] No error messages during startup +- [ ] Worker continues running without crashing +- [ ] When you press `Ctrl+C`, message appears: "Heimdall worker shutting down gracefully..." +- [ ] Process exits cleanly (returns to command prompt) + +**Notes for Tester:** +_Record startup messages:_ + +--- + +### Scenario 6: Job Queue Testing + +**Goal:** Verify that jobs can be queued and processed by the worker. + +**Prerequisites:** Scenario 5 must work (worker can start). + +**Steps:** + +1. Open **two terminal windows** side by side +2. In Terminal 1, start the worker: + ``` + npx heimdall start + ``` +3. Wait for the "polling for jobs" message +4. In Terminal 2, enqueue a test job: + ``` + npx heimdall test-queue + ``` +5. Watch Terminal 1 for job processing +6. Stop the worker in Terminal 1 with `Ctrl+C` + +**Expected Results:** + +- [ ] Terminal 2 shows: "Test job enqueued: test-job-{some-id}" +- [ ] Within 60 seconds, Terminal 1 shows: "Job executed: test-job-{same-id}" +- [ ] No errors in either terminal +- [ ] Job ID in Terminal 2 matches the ID in Terminal 1 + +**Notes for Tester:** +_Record time between enqueueing and execution:_ +_Record job ID:_ + +--- + +### Scenario 7: Test Email Sending + +**Goal:** Verify that Heimdall can send emails through Resend. + +**Prerequisites:** +- Scenario 2 completed with valid Resend API key +- You have access to the email inbox specified + +**Steps:** + +1. Send a test email to yourself: + ``` + npx heimdall test-send --to your-email@example.com + ``` + (Replace with your actual email address) +2. Check your email inbox (including spam/junk folder) + +**Expected Results:** + +- [ ] Command shows: "Test email sent: {resend-message-id}" +- [ ] Email arrives in inbox within 2-5 minutes +- [ ] Email subject: "Heimdall Test Email" +- [ ] Email body confirms configuration is working + +**Notes for Tester:** +_Record Resend message ID:_ +_Time until email arrived:_ +_Did email land in spam?:_ + +--- + +### Scenario 8: Health Endpoint (Local) + +**Goal:** Verify the worker's health endpoint responds correctly. + +**Prerequisites:** Worker can start (Scenario 5). + +**Steps:** + +1. Start the worker: + ``` + npx heimdall start + ``` +2. Open a web browser +3. Navigate to: `http://localhost:3000/health` +4. Stop the worker with `Ctrl+C` + +**Expected Results:** + +- [ ] Browser shows JSON response +- [ ] Response contains: `"status": "healthy"` +- [ ] Response contains: `"queue": "connected"` +- [ ] Response contains: `"uptime": {some-number}` + +**Sample Expected Response:** +```json +{ + "status": "healthy", + "queue": "connected", + "uptime": 45 +} +``` + +**Notes for Tester:** +_Copy the actual response here:_ + +--- + +### Scenario 9: Railway Deployment (Optional) + +**Goal:** Deploy Heimdall to Railway for production use. + +**Prerequisites:** +- Railway account created +- All previous scenarios pass locally + +**Steps:** + +1. Log into Railway Dashboard +2. Create a new project from the GitHub repository +3. Add environment variables in Railway settings: + - `DATABASE_URL` = your Supabase connection string + - `SUPABASE_URL` = your Supabase project URL + - `SUPABASE_ANON_KEY` = your Supabase anon key + - `RESEND_API_KEY` = your Resend API key + - `ADMIN_EMAIL` = your admin email + - `WORKSPACE_NAME` = your workspace name +4. Deploy the service +5. Once deployed, access the health endpoint at your Railway URL + `/health` + +**Expected Results:** + +- [ ] Deployment completes without build errors +- [ ] Service shows as "Running" in Railway dashboard +- [ ] Health endpoint at `https://your-app.railway.app/health` returns: + - `"status": "healthy"` + - `"queue": "connected"` +- [ ] Logs show "Heimdall worker started, polling for jobs..." + +**Notes for Tester:** +_Railway deployment URL:_ +_Any deployment warnings or notes:_ + +--- + +## 4. Success Criteria + +### Minimum Requirements for Sign-off + +All of the following must pass for Epic 1 to be accepted: + +| # | Criteria | Scenario | Status | +|---|----------|----------|--------| +| 1 | Project builds successfully | Scenario 1 | [ ] Pass / [ ] Fail | +| 2 | CLI commands are accessible | Scenario 1 | [ ] Pass / [ ] Fail | +| 3 | Configuration file is created correctly | Scenario 2 | [ ] Pass / [ ] Fail | +| 4 | Configuration validation works | Scenario 2 | [ ] Pass / [ ] Fail | +| 5 | Database migration completes | Scenario 3 | [ ] Pass / [ ] Fail | +| 6 | Database migration is idempotent | Scenario 3 | [ ] Pass / [ ] Fail | +| 7 | Supabase API connection validated | Scenario 4 | [ ] Pass / [ ] Fail | +| 8 | Operational DB connection validated | Scenario 4 | [ ] Pass / [ ] Fail | +| 9 | Worker process starts and polls | Scenario 5 | [ ] Pass / [ ] Fail | +| 10 | Worker shuts down gracefully | Scenario 5 | [ ] Pass / [ ] Fail | +| 11 | Test jobs are processed | Scenario 6 | [ ] Pass / [ ] Fail | +| 12 | Jobs processed within 60 seconds | Scenario 6 | [ ] Pass / [ ] Fail | +| 13 | Test emails are sent and received | Scenario 7 | [ ] Pass / [ ] Fail | +| 14 | Health endpoint responds correctly | Scenario 8 | [ ] Pass / [ ] Fail | + +### Optional (Recommended) + +| # | Criteria | Scenario | Status | +|---|----------|----------|--------| +| 15 | Railway deployment successful | Scenario 9 | [ ] Pass / [ ] Fail / [ ] Skipped | +| 16 | Production health endpoint works | Scenario 9 | [ ] Pass / [ ] Fail / [ ] Skipped | + +--- + +## 5. Known Limitations + +The following are expected behaviors, not bugs: + +1. **Error messages for invalid credentials** are intentionally detailed to help debugging +2. **First database migration** may take 10-30 seconds as pg-boss creates multiple tables +3. **Test emails** may land in spam for unverified domains +4. **Worker polling interval** is 5 seconds, so jobs may take up to 5 seconds to start processing +5. **Config validate** shows partial secrets (first/last 4 characters) for verification purposes + +--- + +## 6. Troubleshooting Guide + +### Common Issues and Solutions + +| Symptom | Likely Cause | Solution | +|---------|--------------|----------| +| "Command not found: heimdall" | Build not completed | Run `npm run build` first | +| "Cannot connect to database" | Wrong db_url | Check Supabase connection string format | +| "Invalid API key" | Resend key incorrect | Regenerate key in Resend dashboard | +| "ECONNREFUSED" | Database not accessible | Check Supabase project is running, check IP restrictions | +| Worker crashes on startup | Missing configuration | Run `heimdall config validate` first | +| Email not received | Domain not verified | Verify domain in Resend or check spam folder | + +--- + +## 7. Sign-off Section + +### Testing Summary + +| Item | Value | +|------|-------| +| **Tester Name** | _________________________ | +| **Test Date** | _________________________ | +| **Environment** | [ ] Local / [ ] Railway / [ ] Both | +| **Node.js Version** | _________________________ | +| **Operating System** | _________________________ | + +### Test Results Summary + +| Category | Passed | Failed | Skipped | +|----------|--------|--------|---------| +| Required Scenarios (1-8) | ___ / 8 | ___ | ___ | +| Optional Scenarios (9) | ___ / 1 | ___ | ___ | +| Success Criteria (1-14) | ___ / 14 | ___ | ___ | + +### Issues Found + +| Issue # | Scenario | Description | Severity | +|---------|----------|-------------|----------| +| | | | [ ] Blocker / [ ] Major / [ ] Minor | +| | | | [ ] Blocker / [ ] Major / [ ] Minor | +| | | | [ ] Blocker / [ ] Major / [ ] Minor | + +### Final Decision + +- [ ] **APPROVED** - All required criteria pass, ready for production +- [ ] **APPROVED WITH CONDITIONS** - Minor issues noted, can proceed +- [ ] **NOT APPROVED** - Blocker issues must be resolved + +### Signatures + +| Role | Name | Signature | Date | +|------|------|-----------|------| +| **Tester** | | | | +| **Product Owner** | | | | +| **Technical Lead** | | | | + +--- + +## 8. Appendix + +### A. CLI Command Reference + +| Command | Purpose | +|---------|---------| +| `heimdall --version` | Display version number | +| `heimdall --help` | Show available commands | +| `heimdall config init` | Create configuration file | +| `heimdall config validate` | Validate configuration | +| `heimdall db migrate` | Set up database tables | +| `heimdall db status` | Check database status | +| `heimdall start` | Start worker process | +| `heimdall test-queue` | Enqueue a test job | +| `heimdall test-send --to EMAIL` | Send a test email | + +### B. Configuration File Template + +```yaml +# heimdall.config.yaml +workspace: + name: "my-workspace" + admin_email: "admin@example.com" + +supabase: + url: "https://xxxxx.supabase.co" + anon_key: "eyJ..." + db_url: "postgresql://postgres:password@db.xxxxx.supabase.co:5432/postgres" + +resend: + api_key: "re_..." + +# Optional AI configuration +ai: + provider: "anthropic" + api_key: "sk-ant-..." + model: "claude-3-haiku-20240307" +``` + +### C. Environment Variables for Railway + +| Variable | Required | Description | +|----------|----------|-------------| +| `DATABASE_URL` | Yes | PostgreSQL connection string | +| `SUPABASE_URL` | Yes | Supabase project URL | +| `SUPABASE_ANON_KEY` | Yes | Supabase anonymous key | +| `RESEND_API_KEY` | Yes | Resend API key | +| `ADMIN_EMAIL` | Yes | Admin notification email | +| `WORKSPACE_NAME` | No | Workspace identifier | +| `PORT` | No | Server port (default: 3000) | + +--- + +*Document generated: January 2, 2026* +*Epic 1: Foundation, CLI & Deployment Infrastructure* diff --git a/scripts/epic-chain.sh b/scripts/epic-chain.sh index 29b0ef683..e7e9a89f1 100755 --- a/scripts/epic-chain.sh +++ b/scripts/epic-chain.sh @@ -9,15 +9,25 @@ # ./epic-chain.sh 36 37 38 --dry-run --verbose # ./epic-chain.sh 36 37 38 --analyze-only # ./epic-chain.sh 36 37 38 --start-from 37 +# ./epic-chain.sh 36 37 38 --uat-gate=full --uat-blocking # # Options: -# --dry-run Show what would be executed without running -# --analyze-only Run analysis phase only, don't execute -# --verbose Show detailed output -# --start-from ID Start from a specific epic (skip earlier ones) -# --skip-done Skip epics/stories with Status: Done -# --no-handoff Don't generate context handoffs between epics -# --no-combined-uat Skip combined UAT generation at end +# --dry-run Show what would be executed without running +# --analyze-only Run analysis phase only, don't execute +# --verbose Show detailed output +# --start-from ID Start from a specific epic (skip earlier ones) +# --skip-done Skip epics/stories with Status: Done +# --no-handoff Don't generate context handoffs between epics +# --no-combined-uat Skip combined UAT generation at end +# +# UAT Gate Options: +# --uat-gate=MODE UAT validation mode: quick|full|skip (default: quick) +# --uat-blocking Halt chain if UAT fails (default: continue) +# --uat-retries=N Max fix attempts per epic (default: 2) +# --no-uat Disable UAT validation gate entirely +# +# Report Options: +# --no-report Skip chain execution report generation # set -e @@ -48,6 +58,19 @@ CYAN='\033[0;36m' BOLD='\033[1m' NC='\033[0m' # No Color +# UAT Gate Configuration +UAT_GATE_ENABLED="${UAT_GATE_ENABLED:-true}" +UAT_GATE_MODE="${UAT_GATE_MODE:-quick}" +UAT_MAX_RETRIES="${UAT_MAX_RETRIES:-2}" +UAT_BLOCKING="${UAT_BLOCKING:-false}" + +# Metrics Configuration +METRICS_DIR="$SPRINT_ARTIFACTS_DIR/metrics" + +# Report Configuration +GENERATE_REPORT="${GENERATE_REPORT:-true}" +CHAIN_REPORT_FILE="$SPRINT_ARTIFACTS_DIR/chain-execution-report.md" + # ============================================================================= # Helper Functions # ============================================================================= @@ -87,6 +110,80 @@ log_section() { echo -e "${BOLD}───────────────────────────────────────────────────────────${NC}" } +# Helper function to create basic report if Claude fails +create_basic_report() { + local end_time_iso=$(date -u +"%Y-%m-%dT%H:%M:%SZ") + local duration_formatted="${DURATION}s" + if [ $DURATION -gt 3600 ]; then + duration_formatted="$((DURATION / 3600))h $((DURATION % 3600 / 60))m" + elif [ $DURATION -gt 60 ]; then + duration_formatted="$((DURATION / 60))m $((DURATION % 60))s" + fi + + cat > "$CHAIN_REPORT_FILE" << EOF +# Epic Chain Execution Report + +## Executive Summary + +**Execution Method:** BMAD Epic Chain (automated AI-driven development) +**Status:** $([ $FAILED_EPICS -eq 0 ] && echo "COMPLETE" || echo "PARTIAL") + +| Metric | Value | +|--------|-------| +| Total Epics | ${#EPIC_IDS[@]} | +| Completed | $COMPLETED_EPICS | +| Failed | $FAILED_EPICS | +| Skipped | $SKIPPED_EPICS | +| Duration | $duration_formatted | + +--- + +## Timeline + +| Epic | Status | +|------|--------| +EOF + + for epic_id in "${EPIC_IDS[@]}"; do + local status="Unknown" + local metrics_file="$METRICS_DIR/epic-${epic_id}-metrics.yaml" + if [ -f "$metrics_file" ]; then + if command -v yq >/dev/null 2>&1; then + local completed=$(yq '.stories.completed // 0' "$metrics_file") + local failed=$(yq '.stories.failed // 0' "$metrics_file") + if [ "$failed" -gt 0 ]; then + status="Partial ($completed completed, $failed failed)" + else + status="Complete ($completed stories)" + fi + fi + fi + echo "| Epic $epic_id | $status |" >> "$CHAIN_REPORT_FILE" + done + + cat >> "$CHAIN_REPORT_FILE" << EOF + +--- + +## Artifacts + +| Artifact | Location | +|----------|----------| +| Chain Plan | $CHAIN_PLAN_FILE | +| Metrics | $METRICS_DIR/ | +| UAT Documents | $UAT_DIR/ | +| Handoffs | $HANDOFF_DIR/ | +| Log | $LOG_FILE | + +--- + +*Report generated: $end_time_iso* +*BMAD Method Epic Chain* +EOF + + log_success "Basic report created: $CHAIN_REPORT_FILE" +} + # ============================================================================= # Argument Parsing # ============================================================================= @@ -130,6 +227,26 @@ while [[ $# -gt 0 ]]; do NO_COMBINED_UAT=true shift ;; + --uat-gate=*) + UAT_GATE_MODE="${1#*=}" + shift + ;; + --uat-blocking) + UAT_BLOCKING=true + shift + ;; + --no-uat) + UAT_GATE_ENABLED=false + shift + ;; + --uat-retries=*) + UAT_MAX_RETRIES="${1#*=}" + shift + ;; + --no-report) + GENERATE_REPORT=false + shift + ;; -*) echo "Unknown option: $1" exit 1 @@ -149,15 +266,25 @@ if [ ${#EPIC_IDS[@]} -eq 0 ]; then echo " $0 36 37 38 --dry-run # Show what would happen" echo " $0 36 37 38 --analyze-only # Just analyze, don't execute" echo " $0 36 37 38 --start-from 37 # Resume from epic 37" + echo " $0 36 37 38 --uat-gate=full # Run full UAT validation after each epic" echo "" echo "Options:" - echo " --dry-run Show execution plan without running" - echo " --analyze-only Analyze dependencies only" - echo " --verbose Detailed output" - echo " --start-from ID Start from specific epic" - echo " --skip-done Skip completed stories" - echo " --no-handoff Skip context handoffs between epics" - echo " --no-combined-uat Skip combined UAT at end" + echo " --dry-run Show execution plan without running" + echo " --analyze-only Analyze dependencies only" + echo " --verbose Detailed output" + echo " --start-from ID Start from specific epic" + echo " --skip-done Skip completed stories" + echo " --no-handoff Skip context handoffs between epics" + echo " --no-combined-uat Skip combined UAT at end" + echo "" + echo "UAT Gate Options:" + echo " --uat-gate=MODE UAT validation mode: quick|full|skip (default: quick)" + echo " --uat-blocking Halt chain if UAT fails (default: continue)" + echo " --uat-retries=N Max fix attempts per epic (default: 2)" + echo " --no-uat Disable UAT validation gate entirely" + echo "" + echo "Report Options:" + echo " --no-report Skip chain execution report generation" exit 1 fi @@ -395,6 +522,60 @@ for current_idx in "${!EXECUTION_ORDER[@]}"; do else if $exec_cmd; then log_success "Epic $epic_id completed" + + # Run UAT validation if enabled + if [ "$UAT_GATE_ENABLED" = true ]; then + log_section "UAT Validation Gate: Epic $epic_id" + + uat_cmd="$SCRIPT_DIR/uat-validate.sh $epic_id --gate-mode=$UAT_GATE_MODE --max-retries=$UAT_MAX_RETRIES" + + if [ "$VERBOSE" = true ]; then + uat_cmd="$uat_cmd --verbose" + fi + + log "Running: $uat_cmd" + + # Capture UAT result + uat_output="" + uat_exit_code=0 + uat_output=$($uat_cmd 2>&1) || uat_exit_code=$? + + echo "$uat_output" >> "$LOG_FILE" + + # Parse result signals + if echo "$uat_output" | grep -q "UAT_GATE_RESULT: PASS"; then + log_success "UAT validation passed for Epic $epic_id" + + # Update metrics file if it exists + epic_metrics_file="$METRICS_DIR/epic-${epic_id}-metrics.yaml" + if [ -f "$epic_metrics_file" ] && command -v yq >/dev/null 2>&1; then + yq -i '.validation.gate_executed = true' "$epic_metrics_file" + yq -i '.validation.gate_status = "PASS"' "$epic_metrics_file" + fi + else + log_error "UAT validation failed for Epic $epic_id" + + # Update metrics file if it exists + epic_metrics_file="$METRICS_DIR/epic-${epic_id}-metrics.yaml" + if [ -f "$epic_metrics_file" ] && command -v yq >/dev/null 2>&1; then + yq -i '.validation.gate_executed = true' "$epic_metrics_file" + yq -i '.validation.gate_status = "FAIL"' "$epic_metrics_file" + fi + + # Extract fix attempts from output + fix_attempts=$(echo "$uat_output" | grep -oE "UAT_FIX_ATTEMPTS: [0-9]+" | grep -oE "[0-9]+" || echo "0") + [ "$VERBOSE" = true ] && log "Fix attempts: $fix_attempts" + + if [ "$UAT_BLOCKING" = true ]; then + log_error "UAT blocking enabled - halting chain" + ((FAILED_EPICS++)) + break + else + log_warn "UAT blocking disabled - continuing to next epic" + fi + fi + fi + ((COMPLETED_EPICS++)) # Generate handoff for next epic @@ -408,6 +589,26 @@ for current_idx in "${!EXECUTION_ORDER[@]}"; do log "Generating context handoff: Epic $epic_id → Epic $next_epic" story_count=${EPIC_STORIES_LIST[$current_idx]} + + # Determine UAT validation status for handoff + uat_status="Not executed" + uat_fix_info="" + if [ "$UAT_GATE_ENABLED" = true ]; then + if echo "$uat_output" | grep -q "UAT_GATE_RESULT: PASS"; then + uat_status="PASS" + local fix_count=$(echo "$uat_output" | grep -oE "UAT_FIX_ATTEMPTS: [0-9]+" | grep -oE "[0-9]+" || echo "0") + if [ "$fix_count" -gt 0 ]; then + uat_status="PASS (after $fix_count fix attempts)" + uat_fix_info="Self-healing fixes were applied. Review fix contexts at: +\`docs/sprint-artifacts/uat-fixes/epic-${epic_id}-fix-context-*.md\`" + fi + else + uat_status="FAIL (non-blocking)" + uat_fix_info="UAT validation failed but chain continued (non-blocking mode). +Review failures at: \`docs/sprint-artifacts/uat-fixes/epic-${epic_id}-fix-context-*.md\`" + fi + fi + cat > "$handoff_file" << EOF # Epic $epic_id → Epic $next_epic Handoff @@ -418,6 +619,11 @@ $(date '+%Y-%m-%d %H:%M:%S') Epic $epic_id has been completed. Key context for Epic $next_epic: +### Implementation Status +- **Stories:** Completed via epic-execute workflow +- **UAT Validation:** $uat_status +- **Metrics:** \`$METRICS_DIR/epic-${epic_id}-metrics.yaml\` + ### Patterns Established - Review code changes in Epic $epic_id for established patterns - Check \`docs/stories/${epic_id}-*\` for implementation details @@ -425,9 +631,17 @@ Epic $epic_id has been completed. Key context for Epic $next_epic: ### Files Modified $(git diff --name-only HEAD~${story_count} HEAD 2>/dev/null | head -20 || echo "Unable to determine - check git log") +### UAT Document +- Location: \`docs/uat/epic-${epic_id}-uat.md\` +- Contains test scenarios for regression testing + +$([ -n "$uat_fix_info" ] && echo "### Fix Context +$uat_fix_info") + ### Notes for Next Epic - Continue following patterns established in this epic -- Reference UAT document at \`docs/uat/epic-${epic_id}-uat.md\` for context +- Ensure changes don't break Epic $epic_id functionality +- Reference UAT document for integration points EOF log_success "Handoff saved to: $handoff_file" @@ -508,6 +722,108 @@ EOF log_success "Combined UAT saved to: $combined_uat_file" fi +# ============================================================================= +# Phase 7: Generate Chain Execution Report +# ============================================================================= + +if [ "$GENERATE_REPORT" = true ] && [ "$DRY_RUN" = false ]; then + log_section "Generating Chain Execution Report" + + # Check if metrics files exist + metrics_found=0 + for epic_id in "${EPIC_IDS[@]}"; do + if [ -f "$METRICS_DIR/epic-${epic_id}-metrics.yaml" ]; then + ((metrics_found++)) + fi + done + + if [ $metrics_found -eq 0 ]; then + log_warn "No metrics files found - skipping report generation" + else + log "Found $metrics_found metrics files" + + # Determine workflow path (installed vs source) + WORKFLOW_PATH="" + if [ -d "$BMAD_DIR/bmm/workflows/4-implementation/epic-chain" ]; then + WORKFLOW_PATH="$BMAD_DIR/bmm/workflows/4-implementation/epic-chain" + elif [ -d "$PROJECT_ROOT/src/modules/bmm/workflows/4-implementation/epic-chain" ]; then + WORKFLOW_PATH="$PROJECT_ROOT/src/modules/bmm/workflows/4-implementation/epic-chain" + fi + + # Build report generation prompt + report_prompt="You are Bob, the Scrum Master, generating a chain execution report. + +## Your Task + +Generate a comprehensive chain execution report for the completed epic chain. + +## Configuration + +- Chain Plan: $CHAIN_PLAN_FILE +- Metrics Folder: $METRICS_DIR +- Output File: $CHAIN_REPORT_FILE +- Stories Location: $STORIES_DIR +- UAT Location: $UAT_DIR +- Epics Location: $EPICS_DIR +- Handoffs Location: $HANDOFF_DIR + +## Epics in Chain + +${EPIC_IDS[*]} + +## Process + +1. Read the chain plan file to understand the epic sequence +2. For each epic, load the metrics file from: $METRICS_DIR/epic-{id}-metrics.yaml +3. Aggregate metrics across all epics: + - Total duration + - Story counts (total, completed, failed, skipped) + - UAT gate results + - Issues encountered +4. Generate the report following the template structure + +## Report Structure + +Generate a markdown report with these sections: +- Executive Summary (status, counts, duration) +- Timeline (epic-by-epic execution details) +- What Was Built (brief per-epic summary) +- Issues Encountered (aggregated from metrics) +- UAT Validation Summary (gate results, fix attempts) +- Artifacts Generated (list generated files) +- Conclusion + +## Output + +Write the report to: $CHAIN_REPORT_FILE + +When complete, output exactly: +REPORT_GENERATED: $CHAIN_REPORT_FILE" + + log "Invoking report generator..." + + # Execute report generation + report_result=$(claude --dangerously-skip-permissions -p "$report_prompt" 2>&1) || true + + echo "$report_result" >> "$LOG_FILE" + + if echo "$report_result" | grep -q "REPORT_GENERATED"; then + log_success "Report generated: $CHAIN_REPORT_FILE" + + # Stage report file + git add "$CHAIN_REPORT_FILE" 2>/dev/null || true + else + log_warn "Report generation may not have completed cleanly" + + # If Claude didn't generate it, create a basic report + if [ ! -f "$CHAIN_REPORT_FILE" ]; then + log "Creating basic report from metrics..." + create_basic_report + fi + fi + fi +fi + # ============================================================================= # Summary # ============================================================================= @@ -527,6 +843,10 @@ echo " Artifacts:" echo " - Chain Plan: $CHAIN_PLAN_FILE" echo " - Handoffs: $HANDOFF_DIR/" echo " - UAT Documents: $UAT_DIR/" +echo " - Metrics: $METRICS_DIR/" +if [ -f "$CHAIN_REPORT_FILE" ]; then +echo " - Report: $CHAIN_REPORT_FILE" +fi echo " - Log: $LOG_FILE" echo "" @@ -537,5 +857,12 @@ fi log_success "All epics completed successfully" echo "" -echo "Next step: Review UAT documents and run manual testing" +if [ -f "$CHAIN_REPORT_FILE" ]; then + echo "Next steps:" + echo " 1. Review execution report: $CHAIN_REPORT_FILE" + echo " 2. Run UAT validation for each epic" + echo " 3. Execute manual test scenarios" +else + echo "Next step: Review UAT documents and run manual testing" +fi echo "" diff --git a/scripts/epic-execute.sh b/scripts/epic-execute.sh index ea90acac3..75b9bd9e5 100755 --- a/scripts/epic-execute.sh +++ b/scripts/epic-execute.sh @@ -63,6 +63,121 @@ log_warn() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] [WARN] $1" >> "$LOG_FILE" } +# ============================================================================= +# Metrics Functions +# ============================================================================= + +METRICS_DIR="" +METRICS_FILE="" + +init_metrics() { + METRICS_DIR="$SPRINT_ARTIFACTS_DIR/metrics" + METRICS_FILE="$METRICS_DIR/epic-${EPIC_ID}-metrics.yaml" + mkdir -p "$METRICS_DIR" + + local start_time=$(date -u +"%Y-%m-%dT%H:%M:%SZ") + + cat > "$METRICS_FILE" << EOF +epic_id: "$EPIC_ID" +execution: + start_time: "$start_time" + end_time: "" + duration_seconds: 0 +stories: + total: 0 + completed: 0 + failed: 0 + skipped: 0 +validation: + gate_executed: false + gate_status: "PENDING" + fix_attempts: 0 +issues: [] +EOF + + log "Metrics initialized: $METRICS_FILE" +} + +update_story_metrics() { + local status="$1" # completed|failed|skipped + + if [ -z "$METRICS_FILE" ] || [ ! -f "$METRICS_FILE" ]; then + return + fi + + # Check if yq is available for YAML manipulation + if command -v yq >/dev/null 2>&1; then + case "$status" in + completed) yq -i '.stories.completed += 1' "$METRICS_FILE" ;; + failed) yq -i '.stories.failed += 1' "$METRICS_FILE" ;; + skipped) yq -i '.stories.skipped += 1' "$METRICS_FILE" ;; + esac + else + # Fallback: log warning (metrics will be finalized at end) + [ "$VERBOSE" = true ] && log_warn "yq not found - metrics update deferred" + fi +} + +add_metrics_issue() { + local story_id="$1" + local issue_type="$2" + local message="$3" + + if [ -z "$METRICS_FILE" ] || [ ! -f "$METRICS_FILE" ]; then + return + fi + + local timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ") + + if command -v yq >/dev/null 2>&1; then + yq -i ".issues += [{\"story\": \"$story_id\", \"type\": \"$issue_type\", \"message\": \"$message\", \"timestamp\": \"$timestamp\"}]" "$METRICS_FILE" + fi +} + +finalize_metrics() { + local total_stories="$1" + local completed="$2" + local failed="$3" + local skipped="$4" + local duration="$5" + + if [ -z "$METRICS_FILE" ] || [ ! -f "$METRICS_FILE" ]; then + return + fi + + local end_time=$(date -u +"%Y-%m-%dT%H:%M:%SZ") + + if command -v yq >/dev/null 2>&1; then + yq -i ".execution.end_time = \"$end_time\"" "$METRICS_FILE" + yq -i ".execution.duration_seconds = $duration" "$METRICS_FILE" + yq -i ".stories.total = $total_stories" "$METRICS_FILE" + yq -i ".stories.completed = $completed" "$METRICS_FILE" + yq -i ".stories.failed = $failed" "$METRICS_FILE" + yq -i ".stories.skipped = $skipped" "$METRICS_FILE" + else + # Fallback: rewrite the file with final values + cat > "$METRICS_FILE" << EOF +epic_id: "$EPIC_ID" +execution: + start_time: "$EPIC_START_TIME" + end_time: "$end_time" + duration_seconds: $duration +stories: + total: $total_stories + completed: $completed + failed: $failed + skipped: $skipped +validation: + gate_executed: false + gate_status: "PENDING" + fix_attempts: 0 +issues: [] +EOF + fi + + log "Metrics finalized: $METRICS_FILE" +} + # ============================================================================= # Argument Parsing # ============================================================================= @@ -142,6 +257,11 @@ log "Project root: $PROJECT_ROOT" mkdir -p "$UAT_DIR" mkdir -p "$SPRINTS_DIR" +# Initialize metrics collection +EPIC_START_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ") +EPIC_START_SECONDS=$(date +%s) +init_metrics + # Find epic file (supports both epic-39-*.md and epic-039-*.md formats) EPIC_FILE="" # Pad epic ID with leading zero for 3-digit format (e.g., 40 -> 040) @@ -554,6 +674,7 @@ for story_file in "${STORIES[@]}"; do else log_warn "Skipping $story_id (waiting for $START_FROM)" ((SKIPPED++)) + update_story_metrics "skipped" continue fi fi @@ -563,6 +684,7 @@ for story_file in "${STORIES[@]}"; do if grep -q "^Status:.*Done" "$story_file" 2>/dev/null; then log_warn "Skipping $story_id (Status: Done)" ((SKIPPED++)) + update_story_metrics "skipped" continue fi fi @@ -576,22 +698,27 @@ for story_file in "${STORIES[@]}"; do if ! execute_dev_phase "$story_file"; then log_error "Dev phase failed for $story_id" ((FAILED++)) + update_story_metrics "failed" + add_metrics_issue "$story_id" "dev_phase_failed" "Development phase did not complete" continue fi - + # REVIEW PHASE (Context 2 - Fresh) if [ "$SKIP_REVIEW" = false ]; then if ! execute_review_phase "$story_file"; then log_error "Review phase failed for $story_id" ((FAILED++)) + update_story_metrics "failed" + add_metrics_issue "$story_id" "review_failed" "Code review phase failed" continue fi fi - + # COMMIT commit_story "$story_id" - + ((COMPLETED++)) + update_story_metrics "completed" log_success "Story complete: $story_id ($COMPLETED/${#STORIES[@]})" done @@ -613,6 +740,9 @@ generate_uat END_TIME=$(date +%s) DURATION=$((END_TIME - START_TIME)) +# Finalize metrics with final counts +finalize_metrics "${#STORIES[@]}" "$COMPLETED" "$FAILED" "$SKIPPED" "$DURATION" + echo "" log "==========================================" log "EPIC EXECUTION COMPLETE" @@ -628,6 +758,7 @@ echo "" echo " Deliverables:" echo " - Stories: $STORIES_DIR/" echo " - UAT: $UAT_DIR/epic-${EPIC_ID}-uat.md" +echo " - Metrics: $METRICS_FILE" echo " - Log: $LOG_FILE" echo "" diff --git a/scripts/uat-validate.sh b/scripts/uat-validate.sh new file mode 100755 index 000000000..422cc79fd --- /dev/null +++ b/scripts/uat-validate.sh @@ -0,0 +1,827 @@ +#!/bin/bash +# +# BMAD UAT Validate - Automated UAT Scenario Execution with Self-Healing Fix Loop +# +# Usage: ./uat-validate.sh [options] +# +# Options: +# --gate-mode=MODE Validation mode: quick|full|skip (default: quick) +# --max-retries=N Max fix attempts before halt (default: 2) +# --skip-manual Skip manual-only scenarios (default: skip) +# --verbose Show detailed output +# --dry-run Show what would be executed without running +# --timeout=SECONDS Timeout per scenario (default: 30) +# +# Exit Codes: +# 0 - UAT PASS (all automatable scenarios passed) +# 1 - UAT FAIL (fixable, retries remain or self-heal succeeded) +# 2 - UAT FAIL (max retries exceeded) +# + +set -e + +# ============================================================================= +# Section 1: Configuration +# ============================================================================= + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" +BMAD_DIR="$PROJECT_ROOT/.bmad" + +UAT_DIR="$PROJECT_ROOT/docs/uat" +SPRINT_ARTIFACTS_DIR="$PROJECT_ROOT/docs/sprint-artifacts" +METRICS_DIR="$SPRINT_ARTIFACTS_DIR/metrics" +FIX_DIR="$SPRINT_ARTIFACTS_DIR/uat-fixes" +STORIES_DIR="$PROJECT_ROOT/docs/stories" + +LOG_FILE="/tmp/bmad-uat-validate-$$.log" + +# Default configuration +UAT_GATE_MODE="quick" +MAX_RETRIES=2 +SKIP_MANUAL=true +VERBOSE=false +DRY_RUN=false +TIMEOUT_SECONDS=30 + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +BOLD='\033[1m' +NC='\033[0m' # No Color + +# ============================================================================= +# Section 2: Helper Functions +# ============================================================================= + +log() { + echo -e "${BLUE}[UAT]${NC} $1" + echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" >> "$LOG_FILE" +} + +log_success() { + echo -e "${GREEN}[PASS]${NC} $1" + echo "[$(date '+%Y-%m-%d %H:%M:%S')] [PASS] $1" >> "$LOG_FILE" +} + +log_error() { + echo -e "${RED}[FAIL]${NC} $1" + echo "[$(date '+%Y-%m-%d %H:%M:%S')] [FAIL] $1" >> "$LOG_FILE" +} + +log_warn() { + echo -e "${YELLOW}[!]${NC} $1" + echo "[$(date '+%Y-%m-%d %H:%M:%S')] [WARN] $1" >> "$LOG_FILE" +} + +log_section() { + echo "" + echo -e "${BOLD}───────────────────────────────────────────────────────────${NC}" + echo -e "${BOLD} $1${NC}" + echo -e "${BOLD}───────────────────────────────────────────────────────────${NC}" +} + +log_header() { + echo "" + echo -e "${CYAN}${BOLD}═══════════════════════════════════════════════════════════${NC}" + echo -e "${CYAN}${BOLD} $1${NC}" + echo -e "${CYAN}${BOLD}═══════════════════════════════════════════════════════════${NC}" + echo "" +} + +# ============================================================================= +# Section 3: Argument Parsing +# ============================================================================= + +EPIC_ID="" + +while [[ $# -gt 0 ]]; do + case $1 in + --gate-mode=*) + UAT_GATE_MODE="${1#*=}" + shift + ;; + --max-retries=*) + MAX_RETRIES="${1#*=}" + shift + ;; + --skip-manual) + SKIP_MANUAL=true + shift + ;; + --include-manual) + SKIP_MANUAL=false + shift + ;; + --verbose) + VERBOSE=true + shift + ;; + --dry-run) + DRY_RUN=true + shift + ;; + --timeout=*) + TIMEOUT_SECONDS="${1#*=}" + shift + ;; + -*) + echo "Unknown option: $1" + exit 1 + ;; + *) + EPIC_ID="$1" + shift + ;; + esac +done + +if [ -z "$EPIC_ID" ]; then + echo "Usage: $0 [options]" + echo "" + echo "Options:" + echo " --gate-mode=MODE Validation mode: quick|full|skip (default: quick)" + echo " --max-retries=N Max fix attempts before halt (default: 2)" + echo " --skip-manual Skip manual-only scenarios (default)" + echo " --include-manual Include manual scenarios in checklist" + echo " --verbose Detailed output" + echo " --dry-run Show what would be executed" + echo " --timeout=SECONDS Timeout per scenario (default: 30)" + echo "" + echo "Exit Codes:" + echo " 0 - UAT PASS" + echo " 1 - UAT FAIL (fixable)" + echo " 2 - UAT FAIL (max retries exceeded)" + exit 1 +fi + +# Validate gate mode +if [[ ! "$UAT_GATE_MODE" =~ ^(quick|full|skip)$ ]]; then + echo "Invalid gate mode: $UAT_GATE_MODE" + echo "Valid modes: quick, full, skip" + exit 1 +fi + +# ============================================================================= +# Section 4: UAT Document Loading +# ============================================================================= + +load_uat_document() { + local epic_id="$1" + + # Find UAT document (try multiple patterns) + UAT_FILE="" + for pattern in "epic-${epic_id}-uat.md" "epic-0${epic_id}-uat.md" "${epic_id}-uat.md"; do + found=$(find "$UAT_DIR" -name "$pattern" 2>/dev/null | head -1) + if [ -n "$found" ]; then + UAT_FILE="$found" + break + fi + done + + if [ -z "$UAT_FILE" ] || [ ! -f "$UAT_FILE" ]; then + log_error "UAT document not found for Epic $epic_id" + log_error "Searched in: $UAT_DIR" + log_error "Expected: epic-${epic_id}-uat.md" + return 1 + fi + + log "Found UAT document: $UAT_FILE" + + # Validate structure - check for scenarios section + if ! grep -qE "^##.*[Ss]cenario|^##.*[Tt]est|^##.*[Cc]riteria" "$UAT_FILE"; then + log_warn "UAT document may not have standard scenario sections" + fi + + # Count scenario blocks (lines starting with ### or numbered items under Test Scenarios) + SCENARIO_COUNT=$(grep -cE "^###|^[0-9]+\." "$UAT_FILE" 2>/dev/null || echo "0") + log "Found approximately $SCENARIO_COUNT scenario entries" + + return 0 +} + +# ============================================================================= +# Section 5: Scenario Classification +# ============================================================================= + +# Arrays to store classified scenarios +declare -a AUTOMATABLE_SCENARIOS +declare -a SEMI_AUTO_SCENARIOS +declare -a MANUAL_SCENARIOS + +classify_scenarios() { + local uat_file="$1" + + # Reset arrays + AUTOMATABLE_SCENARIOS=() + SEMI_AUTO_SCENARIOS=() + MANUAL_SCENARIOS=() + + # Read the UAT file and extract scenario blocks + local current_scenario="" + local current_name="" + local in_scenario=false + local scenario_num=0 + + while IFS= read -r line; do + # Detect scenario headers (### or numbered items) + if [[ "$line" =~ ^###[[:space:]]*(.*) ]] || [[ "$line" =~ ^([0-9]+)\.[[:space:]]+(.*) ]]; then + # Save previous scenario if exists + if [ -n "$current_scenario" ]; then + classify_single_scenario "$scenario_num" "$current_name" "$current_scenario" + fi + + # Start new scenario + ((scenario_num++)) + if [[ "$line" =~ ^###[[:space:]]*(.*) ]]; then + current_name="${BASH_REMATCH[1]}" + else + current_name="${BASH_REMATCH[2]}" + fi + current_scenario="$line" + in_scenario=true + elif [ "$in_scenario" = true ]; then + # Continue accumulating scenario content + current_scenario+=$'\n'"$line" + fi + done < "$uat_file" + + # Handle last scenario + if [ -n "$current_scenario" ]; then + classify_single_scenario "$scenario_num" "$current_name" "$current_scenario" + fi + + log "Classification complete:" + log " Automatable: ${#AUTOMATABLE_SCENARIOS[@]}" + log " Semi-auto: ${#SEMI_AUTO_SCENARIOS[@]}" + log " Manual: ${#MANUAL_SCENARIOS[@]}" +} + +classify_single_scenario() { + local id="$1" + local name="$2" + local content="$3" + + # Check for automatable indicators + if echo "$content" | grep -qiE 'npx|npm run|yarn|node |curl |wget |pytest|jest|vitest|--version|/health|/api/|exit code|returns [0-9]|\.sh |bash '; then + # Extract command from code block if present + local cmd="" + cmd=$(echo "$content" | grep -oE '`[^`]+`' | head -1 | tr -d '`') + if [ -z "$cmd" ]; then + cmd=$(echo "$content" | grep -oE 'npx [a-zA-Z0-9_-]+.*|npm run [a-zA-Z0-9_:-]+.*|curl [^[:space:]]+.*' | head -1) + fi + AUTOMATABLE_SCENARIOS+=("$id|$name|$cmd") + [ "$VERBOSE" = true ] && log " [AUTO] Scenario $id: $name" + + # Check for semi-automated indicators + elif echo "$content" | grep -qiE 'test-send|email|inbox|check your|verify.*manually|setup.*first|start.*server'; then + SEMI_AUTO_SCENARIOS+=("$id|$name|") + [ "$VERBOSE" = true ] && log " [SEMI] Scenario $id: $name" + + # Everything else is manual + else + MANUAL_SCENARIOS+=("$id|$name|") + [ "$VERBOSE" = true ] && log " [MANUAL] Scenario $id: $name" + fi +} + +# ============================================================================= +# Section 6: Scenario Execution +# ============================================================================= + +# Arrays to store results +declare -a PASSED_SCENARIOS +declare -a FAILED_SCENARIOS +declare -a FAILED_DETAILS + +execute_scenarios() { + local gate_mode="$1" + + # Reset results + PASSED_SCENARIOS=() + FAILED_SCENARIOS=() + FAILED_DETAILS=() + + # Skip mode - pass automatically + if [ "$gate_mode" = "skip" ]; then + log "Gate mode: skip - bypassing scenario execution" + echo "UAT_GATE_RESULT: PASS" + echo "UAT_SCENARIOS_PASSED: 0/0 (skipped)" + return 0 + fi + + # Select scenarios based on gate mode + local scenarios_to_run=() + if [ "$gate_mode" = "quick" ]; then + scenarios_to_run=("${AUTOMATABLE_SCENARIOS[@]}") + elif [ "$gate_mode" = "full" ]; then + scenarios_to_run=("${AUTOMATABLE_SCENARIOS[@]}" "${SEMI_AUTO_SCENARIOS[@]}") + fi + + if [ ${#scenarios_to_run[@]} -eq 0 ]; then + log_warn "No automatable scenarios found - gate passes by default" + echo "UAT_GATE_RESULT: PASS" + echo "UAT_SCENARIOS_PASSED: 0/0 (none automatable)" + return 0 + fi + + log_section "Executing ${#scenarios_to_run[@]} Scenarios" + + for scenario_entry in "${scenarios_to_run[@]}"; do + IFS='|' read -r scenario_id scenario_name scenario_cmd <<< "$scenario_entry" + + execute_single_scenario "$scenario_id" "$scenario_name" "$scenario_cmd" + done + + # Report results + local total=${#scenarios_to_run[@]} + local passed=${#PASSED_SCENARIOS[@]} + local failed=${#FAILED_SCENARIOS[@]} + + echo "" + log "Results: $passed/$total passed" + + if [ $failed -eq 0 ]; then + return 0 + else + return 1 + fi +} + +execute_single_scenario() { + local scenario_id="$1" + local scenario_name="$2" + local scenario_cmd="$3" + + echo "" + log "Scenario $scenario_id: $scenario_name" + + # If no command extracted, try to infer from name + if [ -z "$scenario_cmd" ]; then + log_warn " No command detected - marking as manual verification needed" + FAILED_SCENARIOS+=("$scenario_id") + FAILED_DETAILS+=("$scenario_id|$scenario_name|No automatable command found|manual|1") + return 1 + fi + + if [ "$VERBOSE" = true ]; then + log " Command: $scenario_cmd" + fi + + if [ "$DRY_RUN" = true ]; then + echo " [DRY RUN] Would execute: $scenario_cmd" + PASSED_SCENARIOS+=("$scenario_id") + return 0 + fi + + # Execute with timeout + local start_time=$(date +%s%N) + local output="" + local exit_code=0 + local stderr_file="/tmp/uat-stderr-$$.txt" + + # Run command with timeout + set +e + if command -v timeout >/dev/null 2>&1; then + output=$(timeout "$TIMEOUT_SECONDS" bash -c "$scenario_cmd" 2>"$stderr_file") + exit_code=$? + # timeout returns 124 on timeout + if [ $exit_code -eq 124 ]; then + exit_code=124 + fi + else + # macOS fallback using perl + output=$(perl -e 'alarm shift @ARGV; exec @ARGV' "$TIMEOUT_SECONDS" bash -c "$scenario_cmd" 2>"$stderr_file") + exit_code=$? + fi + set -e + + local end_time=$(date +%s%N) + local duration_ms=$(( (end_time - start_time) / 1000000 )) + + local stderr="" + [ -f "$stderr_file" ] && stderr=$(cat "$stderr_file") + rm -f "$stderr_file" + + # Evaluate result + if [ $exit_code -eq 0 ]; then + log_success " Scenario $scenario_id: PASS (${duration_ms}ms)" + PASSED_SCENARIOS+=("$scenario_id") + echo "[$(date '+%Y-%m-%d %H:%M:%S')] Scenario $scenario_id PASS: $scenario_cmd" >> "$LOG_FILE" + elif [ $exit_code -eq 124 ]; then + log_error " Scenario $scenario_id: FAIL (timeout after ${TIMEOUT_SECONDS}s)" + FAILED_SCENARIOS+=("$scenario_id") + FAILED_DETAILS+=("$scenario_id|$scenario_name|$scenario_cmd|timeout|$exit_code|$output|$stderr") + else + log_error " Scenario $scenario_id: FAIL (exit code $exit_code)" + if [ -n "$stderr" ] && [ "$VERBOSE" = true ]; then + echo " Error: $stderr" + fi + FAILED_SCENARIOS+=("$scenario_id") + FAILED_DETAILS+=("$scenario_id|$scenario_name|$scenario_cmd|error|$exit_code|$output|$stderr") + fi + + return $exit_code +} + +# ============================================================================= +# Section 7: Gate Evaluation +# ============================================================================= + +evaluate_gate() { + local total=${#AUTOMATABLE_SCENARIOS[@]} + local passed=${#PASSED_SCENARIOS[@]} + local failed=${#FAILED_SCENARIOS[@]} + + log_section "Gate Evaluation" + + if [ $failed -eq 0 ]; then + log_success "All automatable scenarios passed" + return 0 + else + log_error "$failed scenario(s) failed" + return 1 + fi +} + +# ============================================================================= +# Section 8: Self-Healing Loop +# ============================================================================= + +generate_fix_context() { + local epic_id="$1" + local attempt="$2" + + mkdir -p "$FIX_DIR" + + local fix_file="$FIX_DIR/epic-${epic_id}-fix-context-${attempt}.md" + local timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ") + + # Find template + local template="$PROJECT_ROOT/src/modules/bmm/workflows/5-validation/uat-validate/uat-fix-context-template.md" + + if [ -f "$template" ]; then + # Render template with basic variable substitution + sed -e "s/{epic_id}/$epic_id/g" \ + -e "s/{attempt}/$attempt/g" \ + -e "s/{timestamp}/$timestamp/g" \ + -e "s/{max_retries}/$MAX_RETRIES/g" \ + -e "s/{next_attempt}/$((attempt + 1))/g" \ + -e "s/{failure_count}/${#FAILED_SCENARIOS[@]}/g" \ + -e "s|{uat_doc_path}|$UAT_FILE|g" \ + "$template" > "$fix_file" + else + # Create minimal fix context without template + cat > "$fix_file" << EOF +# UAT Fix Context - Epic $epic_id (Attempt $attempt) + +**Generated:** $timestamp +**Epic:** $epic_id +**Gate Result:** FAIL (${#PASSED_SCENARIOS[@]}/${#AUTOMATABLE_SCENARIOS[@]} scenarios passed) + +--- + +## Summary + +This document contains the context needed to fix UAT failures for Epic $epic_id. + +**Failures to fix:** ${#FAILED_SCENARIOS[@]} +**Fix attempt:** $attempt of $MAX_RETRIES + +--- + +EOF + fi + + # Append failed scenarios details + echo "" >> "$fix_file" + echo "## Failed Scenarios" >> "$fix_file" + echo "" >> "$fix_file" + + for detail in "${FAILED_DETAILS[@]}"; do + IFS='|' read -r scenario_id scenario_name cmd error_type exit_code output stderr <<< "$detail" + + cat >> "$fix_file" << EOF +### Scenario $scenario_id: $scenario_name + +**Command Executed:** +\`\`\`bash +$cmd +\`\`\` + +**Error Type:** $error_type +**Exit Code:** $exit_code + +**Output:** +\`\`\` +$output +\`\`\` + +**Error Output:** +\`\`\` +$stderr +\`\`\` + +--- + +EOF + done + + # Add context references section + cat >> "$fix_file" << EOF + +## Context References + +The following files provide additional context for fixing these failures: + +| File | Purpose | +|------|---------| +| \`$UAT_FILE\` | Full UAT document with all scenarios | +| \`$STORIES_DIR/${epic_id}-*\` | Story files with acceptance criteria | +| \`$METRICS_DIR/epic-${epic_id}-metrics.yaml\` | Execution metrics | + +## Fix Instructions + +Address the failures above in priority order. For each fix: + +1. **Analyze** - Understand why the scenario failed +2. **Locate** - Find the relevant code files +3. **Fix** - Implement the minimum change to resolve the failure +4. **Verify** - Run the scenario command locally to confirm fix +5. **Commit** - Use message format: \`fix(epic-$epic_id): {description}\` + +### Constraints + +- Only fix the identified failures - do not refactor unrelated code +- Run the specific failing commands to verify each fix +- Run project tests after all fixes: \`npm test\` +- If a fix requires changes that would break other scenarios, document the tradeoff + +## After Fixing + +Once all fixes are committed, the UAT validation will automatically re-run. + +- **If all pass:** Epic continues to next phase +- **If failures remain:** Another fix context will be generated (attempt $((attempt + 1))) +- **If max retries exceeded:** Chain halts for human intervention + +--- + +*Generated by UAT Validate Workflow* +*BMAD Method - Epic Chain Self-Healing* +*Fix Context: epic-${epic_id}-fix-context-${attempt}.md* +EOF + + log "Fix context generated: $fix_file" + echo "$fix_file" +} + +run_quick_dev_fix() { + local fix_context_file="$1" + local epic_id="$2" + local attempt="$3" + + log "Spawning quick-dev fix session (attempt $attempt/$MAX_RETRIES)" + + local fix_prompt="You are Barry, the Quick Flow Solo Dev. + +Load and process this fix context document: +$fix_context_file + +Your task: +1. Read the failed scenarios and error details from the fix context +2. Analyze root cause for each failure +3. Implement targeted fixes +4. Run the failing commands to verify fixes +5. Stage changes: git add -A +6. Commit with message: fix(epic-${epic_id}): UAT fix #${attempt} + +Constraints: +- Only fix the identified failures +- Do not refactor unrelated code +- Run tests after fixes + +When done, output exactly: +FIX_COMPLETE: {number_fixed}/${#FAILED_SCENARIOS[@]}" + + if [ "$DRY_RUN" = true ]; then + echo "[DRY RUN] Would spawn Claude for fixes with prompt:" + echo " Fix context: $fix_context_file" + return 0 + fi + + # Execute in isolated context + local result + result=$(claude --dangerously-skip-permissions -p "$fix_prompt" 2>&1) || true + + echo "$result" >> "$LOG_FILE" + + if echo "$result" | grep -q "FIX_COMPLETE"; then + log_success "Quick-dev fix session completed" + return 0 + else + log_warn "Quick-dev fix session may not have completed cleanly" + return 1 + fi +} + +self_healing_loop() { + local epic_id="$1" + local attempt=0 + + while [ $attempt -lt $MAX_RETRIES ]; do + ((attempt++)) + + log_section "Self-Healing Fix Loop (Attempt $attempt/$MAX_RETRIES)" + + # Generate fix context + local fix_file + fix_file=$(generate_fix_context "$epic_id" "$attempt") + + # Run quick-dev fix + if ! run_quick_dev_fix "$fix_file" "$epic_id" "$attempt"; then + log_warn "Fix attempt $attempt may have issues" + fi + + # Re-run validation + log "Re-validating after fix attempt $attempt..." + + # Reset and re-execute + PASSED_SCENARIOS=() + FAILED_SCENARIOS=() + FAILED_DETAILS=() + + if execute_scenarios "$UAT_GATE_MODE"; then + log_success "UAT passed after fix attempt $attempt" + return 0 + fi + + log_warn "UAT still failing after attempt $attempt" + done + + log_error "Max retries ($MAX_RETRIES) exceeded" + return 2 +} + +# ============================================================================= +# Section 9: Output Signals and Metrics +# ============================================================================= + +update_metrics() { + local epic_id="$1" + local gate_status="$2" + local fix_attempts="$3" + + mkdir -p "$METRICS_DIR" + + local metrics_file="$METRICS_DIR/epic-${epic_id}-metrics.yaml" + local timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ") + + # Check if yq is available for YAML manipulation + if command -v yq >/dev/null 2>&1; then + if [ -f "$metrics_file" ]; then + yq -i ".validation.gate_executed = true" "$metrics_file" + yq -i ".validation.gate_status = \"$gate_status\"" "$metrics_file" + yq -i ".validation.fix_attempts = $fix_attempts" "$metrics_file" + yq -i ".validation.scenarios_passed = ${#PASSED_SCENARIOS[@]}" "$metrics_file" + yq -i ".validation.scenarios_failed = ${#FAILED_SCENARIOS[@]}" "$metrics_file" + yq -i ".validation.timestamp = \"$timestamp\"" "$metrics_file" + else + # Create new metrics file + cat > "$metrics_file" << EOF +epic_id: "$epic_id" +validation: + gate_executed: true + gate_status: "$gate_status" + fix_attempts: $fix_attempts + scenarios_passed: ${#PASSED_SCENARIOS[@]} + scenarios_failed: ${#FAILED_SCENARIOS[@]} + timestamp: "$timestamp" +EOF + fi + else + # Fallback: append to file or create new + if [ ! -f "$metrics_file" ]; then + cat > "$metrics_file" << EOF +epic_id: "$epic_id" +validation: + gate_executed: true + gate_status: "$gate_status" + fix_attempts: $fix_attempts + scenarios_passed: ${#PASSED_SCENARIOS[@]} + scenarios_failed: ${#FAILED_SCENARIOS[@]} + timestamp: "$timestamp" +EOF + else + # Simple append for validation section + log_warn "yq not found - metrics update may be incomplete" + fi + fi + + log "Metrics updated: $metrics_file" +} + +output_signals() { + local gate_status="$1" + local fix_attempts="$2" + + local total=${#AUTOMATABLE_SCENARIOS[@]} + local passed=${#PASSED_SCENARIOS[@]} + + echo "" + echo "UAT_GATE_RESULT: $gate_status" + echo "UAT_FIX_ATTEMPTS: $fix_attempts" + echo "UAT_SCENARIOS_PASSED: $passed/$total" +} + +print_summary() { + local gate_status="$1" + local fix_attempts="$2" + + log_header "UAT VALIDATION COMPLETE" + + echo " Epic: $EPIC_ID" + echo " Gate Mode: $UAT_GATE_MODE" + echo " Gate Result: $gate_status" + echo "" + echo " Scenarios:" + echo " Automatable: ${#AUTOMATABLE_SCENARIOS[@]}" + echo " Semi-automated: ${#SEMI_AUTO_SCENARIOS[@]}" + echo " Manual: ${#MANUAL_SCENARIOS[@]}" + echo "" + echo " Results:" + echo " Passed: ${#PASSED_SCENARIOS[@]}" + echo " Failed: ${#FAILED_SCENARIOS[@]}" + echo " Fix Attempts: $fix_attempts" + echo "" + echo " Artifacts:" + echo " Log: $LOG_FILE" + echo " UAT Document: $UAT_FILE" + if [ ${#FAILED_SCENARIOS[@]} -gt 0 ] && [ -d "$FIX_DIR" ]; then + echo " Fix Contexts: $FIX_DIR/" + fi + echo "" +} + +# ============================================================================= +# Main Execution +# ============================================================================= + +log_header "UAT VALIDATION: Epic $EPIC_ID" +log "Gate mode: $UAT_GATE_MODE" +log "Max retries: $MAX_RETRIES" +log "Timeout: ${TIMEOUT_SECONDS}s" + +# Ensure directories exist +mkdir -p "$METRICS_DIR" +mkdir -p "$FIX_DIR" + +# Step 1: Load UAT document +log_section "Loading UAT Document" +if ! load_uat_document "$EPIC_ID"; then + echo "UAT_GATE_RESULT: FAIL" + echo "UAT_FIX_ATTEMPTS: 0" + echo "UAT_SCENARIOS_PASSED: 0/0" + exit 1 +fi + +# Step 2: Classify scenarios +log_section "Classifying Scenarios" +classify_scenarios "$UAT_FILE" + +# Step 3: Execute scenarios +if ! execute_scenarios "$UAT_GATE_MODE"; then + # Gate failed - check if we should try self-healing + if [ "$DRY_RUN" = false ] && [ $MAX_RETRIES -gt 0 ]; then + if ! self_healing_loop "$EPIC_ID"; then + # Max retries exceeded + update_metrics "$EPIC_ID" "FAIL" "$MAX_RETRIES" + output_signals "FAIL" "$MAX_RETRIES" + print_summary "FAIL" "$MAX_RETRIES" + exit 2 + fi + else + # No self-healing or dry-run + update_metrics "$EPIC_ID" "FAIL" "0" + output_signals "FAIL" "0" + print_summary "FAIL" "0" + exit 1 + fi +fi + +# Step 4: Gate passed +FINAL_ATTEMPTS=0 +if [ ${#FAILED_SCENARIOS[@]} -gt 0 ]; then + # Passed after retries + FINAL_ATTEMPTS=$((MAX_RETRIES - $(ls -1 "$FIX_DIR"/epic-${EPIC_ID}-fix-context-*.md 2>/dev/null | wc -l) + 1)) +fi + +update_metrics "$EPIC_ID" "PASS" "$FINAL_ATTEMPTS" +output_signals "PASS" "$FINAL_ATTEMPTS" +print_summary "PASS" "$FINAL_ATTEMPTS" + +log_success "UAT validation passed for Epic $EPIC_ID" +exit 0 diff --git a/src/modules/bmm/workflows/4-implementation/epic-chain/templates/epic-metrics-template.yaml b/src/modules/bmm/workflows/4-implementation/epic-chain/templates/epic-metrics-template.yaml index 160981e56..004ed79e2 100644 --- a/src/modules/bmm/workflows/4-implementation/epic-chain/templates/epic-metrics-template.yaml +++ b/src/modules/bmm/workflows/4-implementation/epic-chain/templates/epic-metrics-template.yaml @@ -9,10 +9,10 @@ epic_file: "{epic_file_path}" # Execution timing execution: - start_time: "{start_timestamp}" # ISO 8601 format - end_time: "{end_timestamp}" # ISO 8601 format - duration_seconds: 0 # Calculated from start/end - duration_formatted: "" # Human readable, e.g., "1.5 hours" + start_time: "{start_timestamp}" # ISO 8601 format + end_time: "{end_timestamp}" # ISO 8601 format + duration_seconds: 0 # Calculated from start/end + duration_formatted: "" # Human readable, e.g., "1.5 hours" # Story counts stories: @@ -44,13 +44,13 @@ uat: # Validation gate results validation: gate_executed: false - gate_mode: "quick" # quick | full | skip + gate_mode: "quick" # quick | full | skip timestamp: "" results: passed: 0 failed: 0 skipped: 0 - gate_status: "PENDING" # PASS | FAIL | PENDING | SKIPPED + gate_status: "PENDING" # PASS | FAIL | PENDING | SKIPPED blocking_issues: [] # Self-healing fix loop tracking @@ -76,19 +76,19 @@ issues: [] # Dependencies (from chain plan) dependencies: - requires: [] # Epic IDs this epic depends on - enables: [] # Epic IDs that depend on this epic + requires: [] # Epic IDs this epic depends on + enables: [] # Epic IDs that depend on this epic # Git information git: - commits: 0 # Number of commits for this epic - branch: "" # Branch used (if feature branch) - first_commit: "" # SHA of first commit - last_commit: "" # SHA of last commit + commits: 0 # Number of commits for this epic + branch: "" # Branch used (if feature branch) + first_commit: "" # SHA of first commit + last_commit: "" # SHA of last commit # Estimated token usage tokens: - estimated_calls: 0 # stories * 2 (dev + review) - estimated_input: 0 # estimated_calls * 8000 - estimated_output: 0 # estimated_calls * 4000 - estimated_total: 0 # input + output + estimated_calls: 0 # stories * 2 (dev + review) + estimated_input: 0 # estimated_calls * 8000 + estimated_output: 0 # estimated_calls * 4000 + estimated_total: 0 # input + output diff --git a/src/modules/bmm/workflows/4-implementation/epic-execute/config/default-config.yaml b/src/modules/bmm/workflows/4-implementation/epic-execute/config/default-config.yaml index 4e49ecbce..89cb41e03 100644 --- a/src/modules/bmm/workflows/4-implementation/epic-execute/config/default-config.yaml +++ b/src/modules/bmm/workflows/4-implementation/epic-execute/config/default-config.yaml @@ -7,13 +7,13 @@ execution: # Automatically commit after each story completes auto_commit: true - + # Run tests before transitioning to review phase run_tests_before_review: true - + # Maximum retries for failed phases max_retries: 2 - + # Timeout per phase in seconds (0 = no timeout) phase_timeout: 0 @@ -24,21 +24,21 @@ review: # standard - Block on acceptance criteria, security, major quality # strict - Block on any quality issue, no auto-fixing mode: standard - + # Allow reviewer to auto-fix issues auto_fix_enabled: true - + # Require all tests pass before approval require_passing_tests: true - + # Issue fix policy thresholds fix_policy: # Always fix HIGH severity issues always_fix_high: true - + # Fix MEDIUM severity if total issues exceed this threshold medium_fix_threshold: 5 - + # Never auto-fix LOW severity (just document) fix_low: false @@ -46,13 +46,13 @@ review: context: # Clear context between phases (recommended) isolate_phases: true - + # Include architecture doc in dev context include_architecture: true - + # Include PRD in dev context include_prd: false - + # Max lines of code context to include max_code_context: 500 @@ -60,16 +60,16 @@ context: uat: # Generate UAT after all stories complete enabled: true - + # UAT output location (relative to project root) output_dir: docs/uat - + # Include edge case scenarios include_edge_cases: true - + # Minimum scenarios to generate min_scenarios: 3 - + # Maximum scenarios to generate max_scenarios: 10 @@ -77,13 +77,13 @@ uat: git: # Commit message prefix commit_prefix: "feat" - + # Include epic ID in commit message include_epic_in_commit: true - + # Push after each commit auto_push: false - + # Branch naming pattern (use {epic_id} and {story_id} as placeholders) branch_pattern: "feature/epic-{epic_id}" @@ -91,10 +91,10 @@ git: parallel: # Enable parallel story execution enabled: false - + # Maximum concurrent stories max_concurrent: 3 - + # Only parallelize independent stories (no dependencies) respect_dependencies: true @@ -102,13 +102,13 @@ parallel: logging: # Log level: debug | info | warn | error level: info - + # Save execution log save_log: true - + # Log output location log_dir: docs/sprints - + # Include Claude responses in log (verbose) log_responses: false @@ -116,9 +116,9 @@ logging: notifications: # Notify on completion on_complete: false - + # Notify on failure on_failure: false - + # Notification method: slack | email | none method: none diff --git a/src/modules/bmm/workflows/4-implementation/epic-execute/templates/uat-template.md b/src/modules/bmm/workflows/4-implementation/epic-execute/templates/uat-template.md index e23866a91..0dfa8c5c1 100644 --- a/src/modules/bmm/workflows/4-implementation/epic-execute/templates/uat-template.md +++ b/src/modules/bmm/workflows/4-implementation/epic-execute/templates/uat-template.md @@ -124,8 +124,8 @@ This epic is **successful** when a user can: | Metric | Value | |--------|-------| -| Scenarios Tested | __ / __ | -| Scenarios Passed | __ / __ | +| Scenarios Tested | \_\_ / \_\_ | +| Scenarios Passed | \_\_ / \_\_ | | Critical Issues | | | Major Issues | | | Minor Issues | | diff --git a/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-01-load-uat.md b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-01-load-uat.md new file mode 100644 index 000000000..a8624ebfb --- /dev/null +++ b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-01-load-uat.md @@ -0,0 +1,102 @@ +# Step 1: Load UAT Document + +## Purpose + +Load and validate the UAT document for the specified epic, extracting all test scenarios for classification and execution. + +## Inputs + +| Input | Source | Required | +|-------|--------|----------| +| epic_id | CLI argument | Yes | +| uat_dir | Configuration | Yes (default: `docs/uat`) | + +## Process + +### 1.1 Locate UAT Document + +Search for UAT document using these patterns in order: +1. `{uat_dir}/epic-{epic_id}-uat.md` +2. `{uat_dir}/epic-0{epic_id}-uat.md` (zero-padded) +3. `{uat_dir}/{epic_id}-uat.md` + +**If not found:** Exit with error code 1 and message: +``` +UAT document not found for Epic {epic_id} +Searched in: {uat_dir} +Expected: epic-{epic_id}-uat.md +``` + +### 1.2 Validate Document Structure + +Confirm document contains at least one of these sections: +- `## Test Scenarios` +- `## Acceptance Criteria` +- `## Scenarios` +- `## Success Criteria` + +**Warning if missing:** Log warning but continue (scenarios may be inline) + +### 1.3 Parse Scenarios + +Extract scenario blocks by detecting: +- Headers starting with `###` (individual scenario titles) +- Numbered items `1.`, `2.`, etc. under scenario sections +- Checkbox items `- [ ]` in criteria sections + +For each scenario, extract: +- **Scenario ID**: Numeric index or explicit ID +- **Scenario Name**: Title text +- **Steps**: Given/When/Then or numbered steps +- **Verification Command**: Code block or CLI reference (if present) +- **Expected Result**: Success criteria text + +### 1.4 Build Scenario List + +Create structured list of scenarios: + +```yaml +scenarios: + - id: 1 + name: "Project Initialization" + steps: + - "Run npx heimdall init" + - "Verify config file created" + verification_command: "npx heimdall --version" + expected_result: "displays a version number" + raw_content: | + ### 1. Project Initialization + ... +``` + +## Outputs + +| Output | Location | Description | +|--------|----------|-------------| +| scenario_list | Memory/State | Array of parsed scenario objects | +| scenario_count | Console | Total number of scenarios found | +| uat_file_path | State | Path to loaded UAT document | + +## Completion Signal + +``` +UAT_LOADED: {scenario_count} scenarios from {uat_file_path} +``` + +## Error Handling + +| Error | Action | +|-------|--------| +| File not found | Exit 1 with clear error message | +| Empty file | Exit 1 with "UAT document is empty" | +| No scenarios detected | Log warning, return empty list (gate passes by default) | +| Parse error | Log warning for specific section, continue with partial results | + +## Example Output + +``` +[UAT] Loading UAT document for Epic 1 +[UAT] Found: docs/uat/epic-1-uat.md +[UAT] Parsed 9 scenarios +UAT_LOADED: 9 scenarios from docs/uat/epic-1-uat.md +``` diff --git a/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-02-classify-scenarios.md b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-02-classify-scenarios.md new file mode 100644 index 000000000..d1a307c36 --- /dev/null +++ b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-02-classify-scenarios.md @@ -0,0 +1,156 @@ +# Step 2: Classify Scenarios + +## Purpose + +Categorize each scenario by its executability level to determine which can be automated, which need partial automation, and which require manual verification. + +## Inputs + +| Input | Source | Required | +|-------|--------|----------| +| scenario_list | Step 1 | Yes | + +## Process + +### 2.1 Classification Categories + +| Classification | Description | Gate Behavior | +|----------------|-------------|---------------| +| **Automatable** | Can be fully executed via shell command | Execute and verify | +| **Semi-automated** | Requires setup, then automated verification | Execute with warning | +| **Manual** | Requires human interaction or visual verification | Skip, add to checklist | + +### 2.2 Detect Automatable Scenarios + +Check scenario content for these indicators (case-insensitive): + +**CLI/Command indicators:** +- `npx`, `npm run`, `yarn`, `pnpm` +- `node `, `python `, `ruby ` +- `curl `, `wget `, `http` +- `bash `, `sh `, `./` + +**Test framework indicators:** +- `pytest`, `jest`, `vitest`, `mocha` +- `npm test`, `yarn test` + +**Verification indicators:** +- `--version`, `--help` +- `/health`, `/api/`, `/status` +- `exit code`, `returns 0`, `returns 1` +- `outputs`, `prints`, `displays` + +**Database/Config indicators:** +- `db migrate`, `db status` +- `config validate`, `config check` + +### 2.3 Detect Semi-Automated Scenarios + +Scenarios with commands that require prior setup: + +**Setup-required indicators:** +- "Start the server first" +- "Ensure database is running" +- "In a separate terminal" +- "After deploying" + +**Partial automation indicators:** +- `test-send`, `send test` +- "check your email/inbox" +- "verify in browser" +- Manual setup + automated verification + +### 2.4 Classify as Manual + +Scenarios without detectable automation path: + +**Manual-only indicators:** +- "Railway dashboard", "Vercel dashboard" +- "Open browser", "Navigate to" +- "Visual inspection", "Visually verify" +- "Two terminals", "Side by side" +- "User should see", "Observe that" +- No code blocks or CLI references + +### 2.5 Extract Commands + +For automatable/semi-automated scenarios, extract the verification command: + +1. Look for inline code: `` `command here` `` +2. Look for code blocks: ```bash ... ``` +3. Look for CLI patterns: `npx ...`, `npm run ...`, `curl ...` +4. Look for expected patterns after "Run:" or "Execute:" + +### 2.6 Build Classification Result + +```yaml +classification: + total: 9 + automatable: 6 + semi_automated: 2 + manual: 1 + +automatable_scenarios: + - id: 1 + name: "Project Initialization" + command: "npx heimdall --version" + expected: "displays a version number" + + - id: 3 + name: "Database Migration" + command: "npx heimdall db migrate" + expected: "success message" + +semi_automated_scenarios: + - id: 7 + name: "Email Notification" + command: "curl -X POST localhost:3000/test-send" + expected: "email received" + note: "Requires manual inbox verification" + +manual_scenarios: + - id: 9 + name: "Dashboard Visual Check" + note: "Requires browser inspection" +``` + +## Outputs + +| Output | Location | Description | +|--------|----------|-------------| +| automatable | Array | Scenarios to execute automatically | +| semi_automated | Array | Scenarios needing setup + execution | +| manual | Array | Scenarios requiring human verification | +| classification_summary | Console | Counts per category | + +## Completion Signal + +``` +SCENARIOS_CLASSIFIED: {automatable}/{semi_automated}/{manual} +``` + +## Example Output + +``` +[UAT] Classifying 9 scenarios... + [AUTO] Scenario 1: Project Initialization + [AUTO] Scenario 2: Configuration Setup + [AUTO] Scenario 3: Database Migration + [AUTO] Scenario 4: Connection Validation + [AUTO] Scenario 5: Worker Process Startup + [AUTO] Scenario 6: Job Queue Testing + [SEMI] Scenario 7: Email Notification + [SEMI] Scenario 8: Webhook Delivery + [MANUAL] Scenario 9: Dashboard Visual Check + +SCENARIOS_CLASSIFIED: 6/2/1 +``` + +## Classification Confidence + +For edge cases, use this priority: +1. If command is clearly present → Automatable +2. If command present but requires setup → Semi-automated +3. If no command detected → Manual + +When in doubt, classify as semi-automated (attempts execution, flags for review). diff --git a/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-03-execute-scenarios.md b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-03-execute-scenarios.md new file mode 100644 index 000000000..07cdfde9d --- /dev/null +++ b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-03-execute-scenarios.md @@ -0,0 +1,202 @@ +# Step 3: Execute Scenarios + +## Purpose + +Run automatable scenarios via shell commands, capture results, and determine pass/fail status for each. + +## Inputs + +| Input | Source | Required | +|-------|--------|----------| +| automatable | Step 2 | Yes | +| semi_automated | Step 2 | For full gate mode | +| gate_mode | CLI | Yes (quick/full/skip) | +| timeout | Config | No (default: 30s) | + +## Process + +### 3.1 Filter by Gate Mode + +| Mode | Scenarios Executed | +|------|-------------------| +| `quick` | Automatable only | +| `full` | Automatable + Semi-automated | +| `skip` | None (gate passes automatically) | + +**Skip mode behavior:** +``` +[UAT] Gate mode: skip - bypassing scenario execution +UAT_GATE_RESULT: PASS +UAT_SCENARIOS_PASSED: 0/0 (skipped) +``` + +### 3.2 Handle Empty Scenario List + +If no scenarios to execute: +``` +[UAT] No automatable scenarios found - gate passes by default +UAT_GATE_RESULT: PASS +UAT_SCENARIOS_PASSED: 0/0 (none automatable) +``` + +### 3.3 Execute Each Scenario + +For each scenario in the execution list: + +#### 3.3.1 Extract Command + +```bash +# From scenario object +command="${scenario.command}" + +# If no command extracted, mark as failed +if [ -z "$command" ]; then + status="FAIL" + error="No automatable command found" +fi +``` + +#### 3.3.2 Execute with Timeout + +```bash +# Using GNU timeout +output=$(timeout $TIMEOUT_SECONDS bash -c "$command" 2>&1) +exit_code=$? + +# macOS fallback (perl-based timeout) +output=$(perl -e 'alarm shift @ARGV; exec @ARGV' $TIMEOUT_SECONDS bash -c "$command" 2>&1) +exit_code=$? +``` + +#### 3.3.3 Capture Results + +Record for each execution: +- **stdout**: Command output +- **stderr**: Error output (captured via 2>&1) +- **exit_code**: Process exit code +- **duration_ms**: Execution time in milliseconds +- **status**: PASS or FAIL + +### 3.4 Result Evaluation Rules + +| Condition | Result | Notes | +|-----------|--------|-------| +| Exit code 0 | PASS | Command succeeded | +| Exit code 124 | FAIL (timeout) | Exceeded timeout limit | +| Exit code 127 | FAIL (not found) | Command not found | +| Exit code non-zero | FAIL (error) | Command failed | + +### 3.5 Expected Output Matching (Optional) + +If scenario has `expected_result`, apply flexible matching: + +**Contains match** (default): +```bash +if echo "$output" | grep -qi "$expected"; then + match="true" +fi +``` + +**Exit code only match:** +```bash +# If expected contains "exit 0" or "returns 0" +if [ $exit_code -eq 0 ]; then + match="true" +fi +``` + +**Note:** Output matching is advisory. Primary pass/fail is based on exit code. + +### 3.6 Record Execution Results + +```yaml +execution_results: + - scenario_id: 1 + name: "Project Initialization" + command: "npx heimdall --version" + status: "PASS" + exit_code: 0 + output: "1.0.0" + duration_ms: 1250 + + - scenario_id: 3 + name: "Database Migration" + command: "npx heimdall db migrate" + status: "FAIL" + exit_code: 1 + output: "" + stderr: "Error: relation 'events' does not exist" + duration_ms: 3400 +``` + +### 3.7 Progress Output + +During execution, output progress: + +``` +[UAT] Executing 6 scenarios... + +Scenario 1: Project Initialization + Command: npx heimdall --version + [PASS] (1250ms) + +Scenario 2: Configuration Setup + Command: npx heimdall config validate + [PASS] (890ms) + +Scenario 3: Database Migration + Command: npx heimdall db migrate + [FAIL] (3400ms) + Error: relation 'events' does not exist +``` + +## Outputs + +| Output | Location | Description | +|--------|----------|-------------| +| results | Array | Per-scenario execution results | +| passed_count | State | Number of passed scenarios | +| failed_count | State | Number of failed scenarios | +| failed_details | Array | Details for failed scenarios | + +## Completion Signal + +``` +SCENARIOS_EXECUTED: {passed}/{total} +``` + +## Error Handling + +| Error | Action | +|-------|--------| +| Command not found | Record as FAIL with "command not found" message | +| Timeout exceeded | Record as FAIL with "timeout after Xs" message | +| Permission denied | Record as FAIL with permission error | +| Shell syntax error | Record as FAIL with syntax error details | + +## Example Complete Output + +``` +[UAT] Executing 6 scenarios (gate mode: quick) + +Scenario 1: Project Initialization + [PASS] npx heimdall --version (1250ms) + +Scenario 2: Configuration Setup + [PASS] npx heimdall config validate (890ms) + +Scenario 3: Database Migration + [FAIL] npx heimdall db migrate (3400ms) + Error: relation 'events' does not exist + +Scenario 4: Connection Validation + [PASS] npx heimdall db status (450ms) + +Scenario 5: Worker Process Startup + [PASS] npx heimdall worker --check (2100ms) + +Scenario 6: Job Queue Testing + [PASS] npx heimdall queue test (1800ms) + +SCENARIOS_EXECUTED: 5/6 +``` diff --git a/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-04-evaluate-gate.md b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-04-evaluate-gate.md new file mode 100644 index 000000000..fc89f1e19 --- /dev/null +++ b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-04-evaluate-gate.md @@ -0,0 +1,237 @@ +# Step 4: Evaluate Gate + +## Purpose + +Determine overall pass/fail status based on execution results. If failed and self-healing is enabled, generate fix context and trigger the quick-dev fix loop. + +## Inputs + +| Input | Source | Required | +|-------|--------|----------| +| results | Step 3 | Yes | +| passed_count | Step 3 | Yes | +| failed_count | Step 3 | Yes | +| failed_details | Step 3 | Yes | +| max_retries | CLI | Yes (default: 2) | +| current_attempt | State | Yes (starts at 0) | +| self_heal_enabled | Config | Yes (default: true) | + +## Process + +### 4.1 Check All Results + +``` +if failed_count == 0: + gate_status = PASS + → Skip to Step 5 (Report Results) +else: + gate_status = FAIL + → Continue to 4.2 +``` + +### 4.2 Evaluate Retry Eligibility + +``` +if current_attempt >= max_retries: + → Max retries exceeded, halt + exit_code = 2 + +if self_heal_enabled == false: + → Self-healing disabled, report failure + exit_code = 1 + +→ Continue to 4.3 (Generate Fix Context) +``` + +### 4.3 Generate Fix Context Document + +Create fix context document at: +``` +{sprint_artifacts}/uat-fixes/epic-{epic_id}-fix-context-{attempt}.md +``` + +#### 4.3.1 Load Template + +Load from: `{workflow_path}/uat-fix-context-template.md` + +If template not found, use inline default structure. + +#### 4.3.2 Populate Template Variables + +| Variable | Value | +|----------|-------| +| `{epic_id}` | Epic ID being validated | +| `{attempt}` | Current fix attempt number | +| `{timestamp}` | ISO 8601 timestamp | +| `{max_retries}` | Maximum retry limit | +| `{next_attempt}` | attempt + 1 | +| `{failure_count}` | Number of failed scenarios | +| `{passed}` | Number of passed scenarios | +| `{total}` | Total scenarios executed | + +#### 4.3.3 Append Failed Scenario Details + +For each failed scenario: + +```markdown +### Scenario {id}: {name} + +**Command Executed:** +```bash +{command} +``` + +**Expected Result:** +{expected_result} + +**Actual Result:** +``` +{actual_output} +``` + +**Error Output:** +``` +{stderr} +``` + +**Exit Code:** {exit_code} + +**Related Story:** {story_id} (if determinable) + +**Root Cause Hint:** +{analyze_error_for_hints} + +--- +``` + +#### 4.3.4 Add Fix Instructions + +```markdown +## Fix Instructions + +Address the failures above in priority order. For each fix: + +1. **Analyze** - Understand why the scenario failed +2. **Locate** - Find the relevant code files +3. **Fix** - Implement the minimum change to resolve +4. **Verify** - Run the scenario command locally +5. **Commit** - Use: `fix(epic-{epic_id}): {description}` + +### Constraints + +- Only fix identified failures +- Do not refactor unrelated code +- Run tests after fixes: `npm test` +``` + +### 4.4 Trigger Quick-Dev Fix + +Spawn a fresh Claude session for fixes: + +```bash +fix_prompt="You are Barry, the Quick Flow Solo Dev. + +Load and process this fix context document: +{fix_context_file} + +Your task: +1. Read the failed scenarios and error details +2. Analyze root cause for each failure +3. Implement targeted fixes +4. Run the failing commands to verify fixes +5. Stage changes: git add -A +6. Commit: fix(epic-{epic_id}): UAT fix #{attempt} + +Constraints: +- Only fix identified failures +- Do not refactor unrelated code +- Run tests after fixes + +When done, output: +FIX_COMPLETE: {fixed_count}/{failure_count}" + +claude --dangerously-skip-permissions -p "$fix_prompt" +``` + +### 4.5 Increment and Re-validate + +After fix session completes: + +1. Increment `current_attempt` +2. Return to Step 3 (Execute Scenarios) +3. Re-run validation with fresh results + +``` +current_attempt += 1 +→ Go to Step 3: Execute Scenarios +→ Then return to Step 4: Evaluate Gate +``` + +### 4.6 Gate Decision Flowchart + +``` +┌─────────────────┐ +│ Execute Results │ +└────────┬────────┘ + │ + ▼ + ┌─────────┐ + │All Pass?│──Yes──▶ GATE_PASS (exit 0) + └────┬────┘ + │No + ▼ + ┌──────────────┐ + │ Retries Left?│──No──▶ GATE_FAIL (exit 2) + └──────┬───────┘ Max retries exceeded + │Yes + ▼ + ┌──────────────┐ + │Self-Heal On? │──No──▶ GATE_FAIL (exit 1) + └──────┬───────┘ Fixable but disabled + │Yes + ▼ + ┌──────────────┐ + │Generate Fix │ + │Context Doc │ + └──────┬───────┘ + │ + ▼ + ┌──────────────┐ + │Spawn Quick- │ + │Dev Fix │ + └──────┬───────┘ + │ + ▼ + ┌──────────────┐ + │Re-validate │──────▶ (Back to Execute) + └──────────────┘ +``` + +## Outputs + +| Output | Location | Description | +|--------|----------|-------------| +| gate_status | State | PASS or FAIL | +| fix_context_file | Path | Generated fix context (if failed) | +| fix_attempt | State | Current attempt number | + +## Completion Signal + +``` +GATE_EVALUATED: PASS +``` +or +``` +GATE_EVALUATED: FAIL +FIX_CONTEXT: {fix_context_file} +FIX_ATTEMPT: {attempt}/{max_retries} +``` + +## Error Handling + +| Error | Action | +|-------|--------| +| Template not found | Use inline default template | +| Fix context write fails | Log error, continue without self-healing | +| Claude session fails | Log output, count as failed attempt | +| Fix didn't resolve issue | Increment attempt, retry if allowed | diff --git a/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-05-report-results.md b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-05-report-results.md new file mode 100644 index 000000000..293cfc24c --- /dev/null +++ b/src/modules/bmm/workflows/5-validation/uat-validate/steps/step-05-report-results.md @@ -0,0 +1,235 @@ +# Step 5: Report Results + +## Purpose + +Update metrics files with validation results and output parseable signals for orchestration scripts (epic-chain.sh) to consume. + +## Inputs + +| Input | Source | Required | +|-------|--------|----------| +| gate_status | Step 4 | Yes | +| results | Step 3 | Yes | +| passed_count | Step 3 | Yes | +| failed_count | Step 3 | Yes | +| fix_attempts | State | Yes | +| epic_id | CLI | Yes | +| metrics_dir | Config | Yes | + +## Process + +### 5.1 Update Metrics File + +Update or create metrics file at: +``` +{metrics_dir}/epic-{epic_id}-metrics.yaml +``` + +#### 5.1.1 Using yq (if available) + +```bash +yq -i '.validation.gate_executed = true' "$metrics_file" +yq -i '.validation.gate_status = "PASS"' "$metrics_file" +yq -i '.validation.gate_mode = "quick"' "$metrics_file" +yq -i '.validation.fix_attempts = 0' "$metrics_file" +yq -i '.validation.scenarios_passed = 5' "$metrics_file" +yq -i '.validation.scenarios_failed = 1' "$metrics_file" +yq -i '.validation.timestamp = "2026-01-05T17:30:00Z"' "$metrics_file" +``` + +#### 5.1.2 Fallback (no yq) + +Create or append validation section: + +```yaml +validation: + gate_executed: true + gate_mode: "quick" + timestamp: "2026-01-05T17:30:00Z" + results: + passed: 5 + failed: 1 + skipped: 3 + gate_status: "PASS" + fix_attempts: 0 +``` + +#### 5.1.3 Record Blocking Issues (if failed) + +```yaml +validation: + # ... other fields ... + blocking_issues: + - scenario_id: 3 + name: "Database Migration" + error: "relation 'events' does not exist" + command: "npx heimdall db migrate" +``` + +#### 5.1.4 Record Fix History (if self-healing occurred) + +```yaml +validation: + # ... other fields ... + fix_history: + - attempt: 1 + timestamp: "2026-01-05T17:35:00Z" + failed_scenarios: ["scenario-3"] + fix_context: "docs/sprint-artifacts/uat-fixes/epic-1-fix-context-1.md" + result: "partial" + - attempt: 2 + timestamp: "2026-01-05T17:42:00Z" + failed_scenarios: [] + result: "success" +``` + +### 5.2 Output Parseable Signals + +Print to stdout in format that epic-chain.sh can parse: + +```bash +# Always output these signals +echo "UAT_GATE_RESULT: $gate_status" +echo "UAT_FIX_ATTEMPTS: $fix_attempts" +echo "UAT_SCENARIOS_PASSED: $passed_count/$total_count" +``` + +**Additional signals on failure:** + +```bash +echo "UAT_FIX_REQUIRED: true" +echo "UAT_FIX_CONTEXT: $fix_context_file" +``` + +**Additional signals on max retries:** + +```bash +echo "UAT_MAX_RETRIES: true" +echo "UAT_HALT_CHAIN: true" +``` + +### 5.3 Set Exit Code + +| Status | Exit Code | Meaning | +|--------|-----------|---------| +| PASS | 0 | All scenarios passed | +| FAIL (fixable) | 1 | Failed but retries remain or self-heal disabled | +| FAIL (max retries) | 2 | Max retries exceeded, chain should halt | + +### 5.4 Print Human-Readable Summary + +``` +═══════════════════════════════════════════════════════════ + UAT VALIDATION COMPLETE +═══════════════════════════════════════════════════════════ + + Epic: 1 + Gate Mode: quick + Gate Result: PASS + + Scenarios: + Automatable: 6 + Semi-automated: 2 + Manual: 1 + + Results: + Passed: 6 + Failed: 0 + Fix Attempts: 0 + + Artifacts: + Log: /tmp/bmad-uat-validate-12345.log + UAT Document: docs/uat/epic-1-uat.md + Metrics: docs/sprint-artifacts/metrics/epic-1-metrics.yaml + +═══════════════════════════════════════════════════════════ +``` + +### 5.5 Failure Summary (if applicable) + +``` +═══════════════════════════════════════════════════════════ + UAT VALIDATION FAILED +═══════════════════════════════════════════════════════════ + + Epic: 1 + Gate Mode: quick + Gate Result: FAIL + + Scenarios: + Automatable: 6 + Semi-automated: 2 + Manual: 1 + + Results: + Passed: 5 + Failed: 1 + Fix Attempts: 2 + + Blocking Issues: + - Scenario 3: Database Migration + Error: relation 'events' does not exist + + Fix Context: + docs/sprint-artifacts/uat-fixes/epic-1-fix-context-2.md + + Action Required: + Manual intervention needed - max retries exceeded + +═══════════════════════════════════════════════════════════ +``` + +## Outputs + +| Output | Location | Description | +|--------|----------|-------------| +| Updated metrics | YAML file | Validation results persisted | +| Signals | stdout | Parseable output for orchestration | +| Summary | stdout | Human-readable summary | +| Exit code | Process | 0=pass, 1=fail-fixable, 2=fail-max-retries | + +## Completion Signal + +``` +RESULTS_REPORTED: {metrics_file_path} +``` + +## Signal Reference + +### For Orchestration Scripts + +| Signal | Values | Description | +|--------|--------|-------------| +| `UAT_GATE_RESULT` | PASS, FAIL | Overall gate status | +| `UAT_FIX_ATTEMPTS` | 0-N | Number of fix attempts made | +| `UAT_SCENARIOS_PASSED` | X/Y | Passed/Total ratio | +| `UAT_FIX_REQUIRED` | true | Indicates fixes were needed | +| `UAT_FIX_CONTEXT` | path | Location of fix context doc | +| `UAT_MAX_RETRIES` | true | Max retries exceeded | +| `UAT_HALT_CHAIN` | true | Chain should stop | + +### Parsing in Shell + +```bash +# In epic-chain.sh +uat_output=$("$SCRIPT_DIR/uat-validate.sh" "$epic_id" 2>&1) + +if echo "$uat_output" | grep -q "UAT_GATE_RESULT: PASS"; then + log_success "UAT passed" +else + log_error "UAT failed" + + if echo "$uat_output" | grep -q "UAT_HALT_CHAIN: true"; then + log_error "Halting chain - max retries exceeded" + exit 2 + fi +fi +``` + +## Error Handling + +| Error | Action | +|-------|--------| +| Metrics file write fails | Log warning, continue with signal output | +| yq not available | Use fallback append method | +| Invalid metrics file | Create new file with validation section | diff --git a/src/modules/bmm/workflows/5-validation/uat-validate/uat-fix-context-template.md b/src/modules/bmm/workflows/5-validation/uat-validate/uat-fix-context-template.md index fa9321a92..358c5c011 100644 --- a/src/modules/bmm/workflows/5-validation/uat-validate/uat-fix-context-template.md +++ b/src/modules/bmm/workflows/5-validation/uat-validate/uat-fix-context-template.md @@ -1,7 +1,7 @@ # UAT Fix Context - Epic {epic_id} (Attempt {attempt}) **Generated:** {timestamp} -**Epic:** {epic_name} +**Epic:** {epic_id} **Gate Result:** FAIL ({passed}/{total} scenarios passed) --- @@ -12,101 +12,16 @@ This document contains the context needed to fix UAT failures for Epic {epic_id} **Failures to fix:** {failure_count} **Fix attempt:** {attempt} of {max_retries} +**UAT Document:** `{uat_doc_path}` --- -## Failed Scenarios +## Quick Start -{#each failed_scenario} -### Scenario {scenario_id}: {scenario_name} - -**Command Executed:** -```bash -{command} -``` - -**Expected Result:** -{expected_result} - -**Actual Result:** -``` -{actual_output} -``` - -**Error Output:** -``` -{stderr} -``` - -**Exit Code:** {exit_code} - -**Related Story:** {story_id} - -**Acceptance Criteria:** -{#each acceptance_criteria} -- {criterion} -{/each} - -**Root Cause Hint:** -{root_cause_hint} - -**Files Likely Involved:** -{#each likely_files} -- `{file_path}` -{/each} +1. Read each failed scenario below +2. Identify root cause from error output +3. Locate and fix the relevant code +4. Verify fix: run the failing command +5. Commit: `fix(epic-{epic_id}): {description}` --- -{/each} - -## Fix Instructions - -Address the failures above in priority order. For each fix: - -1. **Analyze** - Understand why the scenario failed -2. **Locate** - Find the relevant code files -3. **Fix** - Implement the minimum change to resolve the failure -4. **Verify** - Run the scenario command locally to confirm fix -5. **Commit** - Use message format: `fix(epic-{epic_id}): {description}` - -### Priority Order - -{#each failed_scenario} -{priority}. **Scenario {scenario_id}**: {one_line_description} -{/each} - ---- - -## Constraints - -- Only fix the identified failures - do not refactor unrelated code -- Run the specific failing commands to verify each fix -- Run project tests after all fixes: `npm test` -- Commit with conventional format: `fix(epic-{epic_id}): {description}` -- If a fix requires changes that would break other scenarios, document the tradeoff - ---- - -## Context Files - -The following files may provide additional context: - -| File | Purpose | -|------|---------| -| `{uat_doc_path}` | Full UAT document with all scenarios | -| `{story_files}` | Story files with complete acceptance criteria | -| `{architecture_doc}` | System architecture reference | - ---- - -## After Fixing - -Once all fixes are committed, the UAT validation will automatically re-run. - -- **If all pass:** Epic continues to next phase -- **If failures remain:** Another fix context will be generated (attempt {next_attempt}) -- **If max retries exceeded:** Chain halts for human intervention - ---- - -*Generated by UAT Validate Workflow* -*BMAD Method - Epic Chain Self-Healing*