---
title: "UAT Workflow Implementation Gaps & Required Fixes"
---

# UAT Workflow Implementation Gaps & Required Fixes

## Executive Summary

The UAT validation workflow and epic chain report generator have been designed and partially implemented, but several critical components are missing to make them production-ready. This document outlines the gaps, required fixes, and context needed to complete the implementation.

**Current State:** Workflow definitions, templates, and agent triggers exist
**Missing:** Shell orchestration, metrics collection, step files, and integration points

---

## Architecture Context

### How BMAD Workflows Achieve Context Isolation

Native BMAD workflows use a **shell script orchestration pattern** to isolate Claude contexts:

```
┌─────────────────────────────────────────────────────────────────────┐
│                    BMAD CONTEXT ISOLATION PATTERN                    │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  Shell Script (epic-execute.sh / epic-chain.sh)                     │
│       │                                                              │
│       ├──► claude -p "Phase 1 prompt..."  ──► Fresh Context A       │
│       │         │                                                    │
│       │         └──► Git staging / Story file (state transfer)      │
│       │                                                              │
│       ├──► claude -p "Phase 2 prompt..."  ──► Fresh Context B       │
│       │         │                                                    │
│       │         └──► Git staging / Story file (state transfer)      │
│       │                                                              │
│       └──► claude -p "Phase 3 prompt..."  ──► Fresh Context C       │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘
```

**Key Files:**
- `scripts/epic-execute.sh` - 600+ lines, handles dev→review→commit→UAT phases
- `scripts/epic-chain.sh` - Handles multi-epic execution with handoffs

**State Transfer Mechanisms:**
1. **Git staging** - Code changes pass between contexts via `git add`
2. **Story files** - Dev Agent Record and Code Review Record sections
3. **Handoff documents** - Cross-epic context transfer
4. **Completion signals** - `IMPLEMENTATION COMPLETE: {id}`, `REVIEW PASSED: {id}`

---

## Gap 1: Missing Shell Orchestration Script

### Problem

The UAT validation self-healing loop requires context isolation:

```
UAT Validate (Quinn) → Fix Context → Quick Dev (Barry) → Re-validate (Quinn)
```

Without a shell script, these phases would run in the same Claude context, defeating the purpose of fresh-eyes validation after fixes.

### Required: `scripts/uat-validate.sh`

**Location:** `scripts/uat-validate.sh`

**Responsibilities:**
1. Load UAT document for specified epic
2. Execute automatable scenarios via shell commands
3. Collect pass/fail results with evidence
4. On failure: Generate fix context document
5. Spawn fresh Claude session for quick-dev fixes
6. Re-validate in fresh context
7. Loop until pass or max retries
8. Output final gate result

**Interface:**
```bash
# Usage
./scripts/uat-validate.sh <epic_id> [options]

# Options
--gate-mode=quick|full|skip    # Which scenarios to run
--max-retries=2                # Fix attempts before halt
--skip-manual                  # Skip manual-only scenarios
--verbose                      # Detailed output

# Output signals (for epic-chain.sh to parse)
UAT_GATE_RESULT: PASS|FAIL
UAT_FIX_ATTEMPTS: N
UAT_SCENARIOS_PASSED: X/Y
```

**Context Flow:**
```
┌─────────────────────────────────────────────────────────────────────┐
│                     uat-validate.sh Flow                             │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  1. Load UAT doc: docs/uat/epic-{id}-uat.md                         │
│                                                                      │
│  2. Classify scenarios (automatable / semi-auto / manual)           │
│                                                                      │
│  3. Execute automatable scenarios via shell:                        │
│     for scenario in automatable:                                    │
│         result = execute_command(scenario.command)                  │
│         record_result(scenario.id, result)                          │
│                                                                      │
│  4. Evaluate gate:                                                  │
│     if all_passed:                                                  │
│         echo "UAT_GATE_RESULT: PASS"                                │
│         exit 0                                                      │
│                                                                      │
│  5. On failure (attempt < max_retries):                             │
│     a. Generate fix context document                                │
│     b. Spawn fresh Claude for quick-dev:                            │
│        claude -p "Load fix context, implement fixes..."             │
│     c. Increment attempt counter                                    │
│     d. Go to step 3 (re-validate)                                   │
│                                                                      │
│  6. On max retries exceeded:                                        │
│     echo "UAT_GATE_RESULT: FAIL"                                    │
│     echo "UAT_MAX_RETRIES: true"                                    │
│     exit 2                                                          │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘
```

**Reference Implementation:** See `scripts/epic-execute.sh` lines 289-438 for how dev and review phases spawn isolated Claude sessions.

---

## Gap 2: Metrics Collection Not Instrumented

### Problem

The report generator expects metrics at `{metrics_folder}/epic-{id}-metrics.yaml`, but `epic-execute.sh` doesn't write these files.

### Required: Metrics Instrumentation in `epic-execute.sh`

**Location:** Modify `scripts/epic-execute.sh`

**Add at epic start (~line 520):**
```bash
# Initialize metrics
EPIC_START_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
METRICS_FILE="$SPRINT_ARTIFACTS/metrics/epic-${EPIC_ID}-metrics.yaml"
mkdir -p "$SPRINT_ARTIFACTS/metrics"

# Initialize metrics file
cat > "$METRICS_FILE" << EOF
epic_id: "$EPIC_ID"
epic_name: "$EPIC_NAME"
execution:
  start_time: "$EPIC_START_TIME"
  end_time: ""
  duration_seconds: 0
stories:
  total: ${#STORIES[@]}
  completed: 0
  failed: 0
  skipped: 0
validation:
  gate_executed: false
  gate_status: "PENDING"
  fix_attempts: 0
issues: []
EOF
```

**Add after each story (~line 595):**
```bash
# Update metrics after story completion
update_story_metrics() {
    local status="$1"  # completed | failed | skipped
    local story_id="$2"

    # Increment appropriate counter
    case "$status" in
        completed) yq -i '.stories.completed += 1' "$METRICS_FILE" ;;
        failed)    yq -i '.stories.failed += 1' "$METRICS_FILE" ;;
        skipped)   yq -i '.stories.skipped += 1' "$METRICS_FILE" ;;
    esac
}
```

**Add at epic end (~line 620):**
```bash
# Finalize metrics
EPIC_END_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
DURATION=$(($(date +%s) - $(date -d "$EPIC_START_TIME" +%s)))

yq -i ".execution.end_time = \"$EPIC_END_TIME\"" "$METRICS_FILE"
yq -i ".execution.duration_seconds = $DURATION" "$METRICS_FILE"
```

**Template Reference:** `src/modules/bmm/workflows/4-implementation/epic-chain/templates/epic-metrics-template.yaml`

---

## Gap 3: UAT Gate Not Integrated Into epic-chain.sh

### Problem

`epic-chain.sh` executes epics but doesn't call UAT validation between them.

### Required: Add UAT Gate Phase to `epic-chain.sh`

**Location:** Modify `scripts/epic-chain.sh`

**Add configuration variables (~line 50):**
```bash
# UAT Gate Configuration
UAT_GATE_ENABLED="${UAT_GATE_ENABLED:-true}"
UAT_GATE_MODE="${UAT_GATE_MODE:-quick}"
UAT_MAX_RETRIES="${UAT_MAX_RETRIES:-2}"
UAT_BLOCKING="${UAT_BLOCKING:-false}"
```

**Add after epic completion (~line 480, after `run_epic_execute`):**
```bash
# Run UAT validation if enabled
if [ "$UAT_GATE_ENABLED" = true ]; then
    log "Running UAT validation for Epic $epic_id..."

    uat_result=$(./scripts/uat-validate.sh "$epic_id" \
        --gate-mode="$UAT_GATE_MODE" \
        --max-retries="$UAT_MAX_RETRIES" 2>&1)

    # Parse result
    if echo "$uat_result" | grep -q "UAT_GATE_RESULT: PASS"; then
        log "✓ UAT validation passed for Epic $epic_id"
        # Update metrics
        yq -i '.validation.gate_executed = true' "$METRICS_FILE"
        yq -i '.validation.gate_status = "PASS"' "$METRICS_FILE"
    else
        log_error "✗ UAT validation failed for Epic $epic_id"
        yq -i '.validation.gate_executed = true' "$METRICS_FILE"
        yq -i '.validation.gate_status = "FAIL"' "$METRICS_FILE"

        if [ "$UAT_BLOCKING" = true ]; then
            log_error "UAT blocking enabled - halting chain"
            exit 1
        else
            log_warn "UAT blocking disabled - continuing to next epic"
        fi
    fi
fi
```

**Add CLI flag parsing (~line 120):**
```bash
--uat-gate=*)
    UAT_GATE_MODE="${1#*=}"
    ;;
--uat-blocking)
    UAT_BLOCKING=true
    ;;
--no-uat)
    UAT_GATE_ENABLED=false
    ;;
```

---

## Gap 4: Step Files Not Following BMAD Pattern

### Problem

The UAT validation workflow has `instructions.md` but not the step-file architecture used by epic-execute:
- `step-01-init.md`
- `step-02-dev-story.md`
- `step-03-code-review.md`
- etc.

### Required: Create Step Files for UAT Validation

**Location:** `src/modules/bmm/workflows/5-validation/uat-validate/steps/`

**Files to create:**

| File | Purpose |
|------|---------|
| `step-01-load-uat.md` | Load UAT document, parse scenarios |
| `step-02-classify-scenarios.md` | Categorize as automatable/semi-auto/manual |
| `step-03-execute-scenarios.md` | Run automatable scenarios via shell |
| `step-04-evaluate-gate.md` | Determine pass/fail, generate fix context if needed |
| `step-05-report-results.md` | Update metrics, output signals |

**Step File Template (from epic-execute pattern):**
```markdown
# Step N: {Title}

## Purpose
{What this step accomplishes}

## Inputs
| Input | Source | Required |
|-------|--------|----------|
| ... | ... | Yes/No |

## Process

### N.1 {Sub-step}
{Detailed instructions}

### N.2 {Sub-step}
{Detailed instructions}

## Outputs
| Output | Location | Description |
|--------|----------|-------------|
| ... | ... | ... |

## Completion Signal
```
{SIGNAL_NAME}: {value}
```

## Error Handling
| Error | Action |
|-------|--------|
| ... | ... |
```

---

## Gap 5: Fix Context Not Integrated With Handoff Pattern

### Problem

The fix context template exists but isn't written to the handoff directory or loaded by quick-dev.

### Required: Integrate Fix Context With Handoff Flow

**Fix context location should follow handoff pattern:**
```
docs/sprint-artifacts/
├── handoffs/
│   ├── epic-1-to-2-handoff.md
│   └── epic-2-to-3-handoff.md
├── uat-fixes/                    # NEW
│   ├── epic-1-fix-context-1.md
│   └── epic-2-fix-context-1.md
└── metrics/
    ├── epic-1-metrics.yaml
    └── epic-2-metrics.yaml
```

**In `uat-validate.sh`, write fix context:**
```bash
generate_fix_context() {
    local epic_id="$1"
    local attempt="$2"
    local failed_scenarios="$3"

    local fix_dir="$SPRINT_ARTIFACTS/uat-fixes"
    mkdir -p "$fix_dir"

    local fix_file="$fix_dir/epic-${epic_id}-fix-context-${attempt}.md"

    # Render template with variables
    sed -e "s/{epic_id}/$epic_id/g" \
        -e "s/{attempt}/$attempt/g" \
        -e "s/{timestamp}/$(date -u +"%Y-%m-%dT%H:%M:%SZ")/g" \
        "$WORKFLOW_PATH/uat-fix-context-template.md" > "$fix_file"

    # Append failed scenarios
    echo "" >> "$fix_file"
    echo "## Failed Scenarios" >> "$fix_file"
    echo "$failed_scenarios" >> "$fix_file"

    echo "$fix_file"
}
```

**In quick-dev fix session, load context:**
```bash
run_quick_dev_fix() {
    local fix_context_file="$1"

    local fix_prompt="You are Barry, the Quick Flow Solo Dev.

Load and process this fix context document:
$fix_context_file

Your task:
1. Read the failed scenarios and error details
2. Analyze root cause for each failure
3. Implement targeted fixes
4. Run the failing commands to verify fixes
5. Commit with message: fix(epic-{id}): UAT fix #{attempt}

Constraints:
- Only fix the identified failures
- Do not refactor unrelated code
- Run tests after fixes

When done, output:
FIX_COMPLETE: {number_fixed}/{total_failures}
"

    # Fresh Claude context for fixes
    claude --dangerously-skip-permissions -p "$fix_prompt"
}
```

---

## Gap 6: Report Generator Missing Chain Integration

### Problem

Step 10 (generate report) exists but isn't called automatically by `epic-chain.sh`.

### Required: Add Report Generation to `epic-chain.sh`

**Add after all epics complete (~line 550):**
```bash
# Generate chain execution report
if [ "$GENERATE_REPORT" = true ]; then
    log "Generating chain execution report..."

    # Invoke report generation via Claude
    report_prompt="You are Bob, the Scrum Master.

Execute the chain report generation workflow:
- Load: $INSTALLED_PATH/steps/step-10-generate-report.md
- Metrics folder: $SPRINT_ARTIFACTS/metrics
- Chain plan: $CHAIN_PLAN_FILE
- Output to: $CHAIN_REPORT_FILE

Generate the complete execution report."

    claude --dangerously-skip-permissions -p "$report_prompt"

    if [ -f "$CHAIN_REPORT_FILE" ]; then
        log "✓ Report generated: $CHAIN_REPORT_FILE"
    else
        log_error "Report generation failed"
    fi
fi
```

---

## Implementation Priority

| Priority | Gap | Effort | Impact |
|----------|-----|--------|--------|
| **P0** | Create `uat-validate.sh` | High | Critical - self-healing loop won't work without it |
| **P0** | Add metrics instrumentation to `epic-execute.sh` | Medium | Critical - report generator needs data |
| **P1** | Integrate UAT gate into `epic-chain.sh` | Medium | High - enables automated validation |
| **P1** | Integrate report generation into `epic-chain.sh` | Low | High - automates report creation |
| **P2** | Create step files for UAT validation | Medium | Medium - improves maintainability |
| **P2** | Integrate fix context with handoff pattern | Low | Medium - cleaner file organization |

---

## File Reference

### Existing Files (Created)

| File | Status | Purpose |
|------|--------|---------|
| `src/modules/bmm/agents/uat-validator.agent.yaml` | ✅ Complete | Quinn agent definition |
| `src/modules/bmm/workflows/5-validation/uat-validate/workflow.yaml` | ✅ Complete | Workflow configuration |
| `src/modules/bmm/workflows/5-validation/uat-validate/instructions.md` | ✅ Complete | Execution instructions |
| `src/modules/bmm/workflows/5-validation/uat-validate/uat-fix-context-template.md` | ✅ Complete | Fix context template |
| `src/modules/bmm/workflows/4-implementation/epic-chain/templates/chain-report-template.md` | ✅ Complete | Report template |
| `src/modules/bmm/workflows/4-implementation/epic-chain/templates/epic-metrics-template.yaml` | ✅ Complete | Metrics schema |
| `src/modules/bmm/workflows/4-implementation/epic-chain/steps/step-10-generate-report.md` | ✅ Complete | Report generation step |

### Files to Create

| File | Priority | Purpose |
|------|----------|---------|
| `scripts/uat-validate.sh` | P0 | Shell orchestration for UAT validation |
| `src/modules/bmm/workflows/5-validation/uat-validate/steps/step-01-load-uat.md` | P2 | Step file |
| `src/modules/bmm/workflows/5-validation/uat-validate/steps/step-02-classify-scenarios.md` | P2 | Step file |
| `src/modules/bmm/workflows/5-validation/uat-validate/steps/step-03-execute-scenarios.md` | P2 | Step file |
| `src/modules/bmm/workflows/5-validation/uat-validate/steps/step-04-evaluate-gate.md` | P2 | Step file |
| `src/modules/bmm/workflows/5-validation/uat-validate/steps/step-05-report-results.md` | P2 | Step file |

### Files to Modify

| File | Priority | Changes |
|------|----------|---------|
| `scripts/epic-execute.sh` | P0 | Add metrics collection |
| `scripts/epic-chain.sh` | P1 | Add UAT gate phase, report generation |

---

## Testing Plan

### Unit Tests

1. **uat-validate.sh** - Test scenario classification, command execution, signal output
2. **Metrics collection** - Verify YAML structure matches template
3. **Report generation** - Verify all placeholders replaced

### Integration Tests

1. **Self-healing loop** - Intentionally fail a scenario, verify fix + re-validate
2. **Epic chain with UAT** - Run full chain, verify UAT runs after each epic
3. **Report accuracy** - Compare report metrics to actual execution

### Manual Verification

1. Run `epic-chain 1-3` on test project
2. Verify metrics files created
3. Verify UAT validation runs
4. Verify report generated with accurate data

---

## Conclusion

The UAT validation and report generation workflows are architecturally sound but missing the shell orchestration layer that makes BMAD workflows production-ready. The highest priority items are:

1. **Create `uat-validate.sh`** - Without this, context isolation doesn't happen
2. **Instrument `epic-execute.sh`** - Without this, reports have no data

Once these are complete, the full flow will work:

```
epic-chain → epic-execute → UAT validate → (fix loop if needed) → next epic → report
```