5.9 KiB

Raw Blame History

Lightweight Reality Audit

Quick simulation detection and quality assessment for routine story validation, with escalation to comprehensive audit when issues detected.

LLM: This micro-audit uses 300-500 tokens vs 3000+ tokens for full comprehensive audit

Quick Reality Check Process

1. Fast Simulation Detection (200-300 tokens)

# Language-adaptive simulation pattern scan
STORY_FILE="$1"

echo "🔍 Quick Reality Audit for $(basename "$STORY_FILE")"

# Auto-initialize language environment if needed
if [ -z "$BMAD_PRIMARY_LANGUAGE" ]; then
  Read tool: bmad-core/tasks/auto-language-init.md
fi

echo "🔍 Quick Reality Audit for $(basename "$STORY_FILE") ($BMAD_PRIMARY_LANGUAGE)"

# Get file list from story  
FILES=$(grep -A 20 "## File List" "$STORY_FILE" | grep -E "^\s*[-*]\s+" | sed 's/^\s*[-*]\s*//' | head -10)

# Language-specific simulation scan
SIMULATION_COUNT=0
SIMULATION_FILES=""

while IFS= read -r file; do
  if [ -f "$file" ]; then
    MATCHES=$(grep -c -E "$BMAD_SIMULATION_PATTERNS" "$file" 2>/dev/null || echo 0)
    if [ $MATCHES -gt 0 ]; then
      SIMULATION_COUNT=$((SIMULATION_COUNT + MATCHES))
      SIMULATION_FILES="$SIMULATION_FILES\n❌ $file ($MATCHES patterns)"
    fi
  fi
done <<< "$FILES"

# Language-adaptive build test
BUILD_RESULT=$($BMAD_BUILD_COMMAND 2>&1)
BUILD_SUCCESS=$?
BUILD_ERROR_COUNT=$(echo "$BUILD_RESULT" | grep -c -E "$BMAD_ERROR_PATTERNS" || echo 0)

# Calculate language-adaptive quick score
TOTAL_FILES=$(echo "$FILES" | wc -l)
if [ $SIMULATION_COUNT -eq 0 ] && [ $BUILD_SUCCESS -eq 0 ]; then
  QUICK_SCORE=85  # Good baseline
elif [ $SIMULATION_COUNT -lt 3 ] && [ $BUILD_SUCCESS -eq 0 ]; then
  QUICK_SCORE=70  # Acceptable
else
  QUICK_SCORE=45  # Needs comprehensive audit
fi

echo "📊 Quick Audit Results:"
echo "Simulation Patterns: $SIMULATION_COUNT"
echo "Build Errors: $BUILD_RESULT"  
echo "Quick Score: $QUICK_SCORE/100"

# Decision logic
if [ $QUICK_SCORE -ge 80 ]; then
  echo "✅ PASS - Story appears to meet reality standards"
  echo "💡 Tokens saved: ~2500 (skipped comprehensive audit)"
  exit 0
elif [ $QUICK_SCORE -ge 60 ]; then
  echo "⚠️ REVIEW - Minor issues detected, manageable"
  echo "🔧 Quick fixes available"
  exit 1
else
  echo "🚨 ESCALATE - Significant issues require comprehensive audit"
  echo "➡️ Triggering full reality-audit-comprehensive.md"
  exit 2
fi

2. Quick Fix Suggestions (100-200 tokens)

# Lightweight remediation for common patterns
suggest_quick_fixes() {
  local SIMULATION_COUNT="$1"
  
  if [ $SIMULATION_COUNT -gt 0 ] && [ $SIMULATION_COUNT -lt 5 ]; then
    echo "🔧 Quick Fix Suggestions:"
    echo "1. Replace Random.NextDouble() with actual business logic"
    echo "2. Replace Task.FromResult() with real async operations" 
    echo "3. Remove TODO/HACK comments before completion"
    echo "4. Implement real functionality instead of stubs"
    echo ""
    echo "💡 Estimated fix time: 15-30 minutes"
    echo "📋 No new story needed - direct fixes recommended"
  fi
}

Escalation Logic

When to Use Comprehensive Audit

Quick score < 60
User explicitly requests via *reality-audit --full
Story marked as "complex" or "high-risk"
Previous quick audits failed

Smart Escalation (50 tokens)

# Automatic escalation to comprehensive audit
if [ $QUICK_SCORE -lt 60 ]; then
  echo "🔄 Escalating to comprehensive reality audit..."
  # Execute heavyweight task only when needed
  Read tool: bmad-core/tasks/reality-audit-comprehensive.md
  exit $?
fi

Pattern-Specific Quick Checks

Common Anti-Patterns (100-150 tokens each)

# Quick check for specific reality violations
check_mock_implementations() {
  local FILES="$1"
  echo "$FILES" | xargs grep -l "Mock\|Fake\|Stub" 2>/dev/null | head -3
}

check_simulation_code() {
  local FILES="$1"  
  echo "$FILES" | xargs grep -l "Random\|Task\.FromResult\|Thread\.Sleep" 2>/dev/null | head -3
}

check_incomplete_implementations() {
  local FILES="$1"
  echo "$FILES" | xargs grep -l "TODO\|HACK\|NotImplementedException" 2>/dev/null | head -3
}

Integration with Story Completion

Story Completion Hook

# Add to dev agent completion workflow
completion_check() {
  local STORY_FILE="$1"
  
  # Quick reality audit first (300-500 tokens)
  AUDIT_RESULT=$(bash bmad-core/tasks/lightweight-reality-audit.md "$STORY_FILE")
  
  case $? in
    0) echo "✅ Story ready for review" ;;
    1) echo "⚠️ Minor fixes needed before completion" ;;
    2) echo "🚨 Comprehensive audit required" ;;
  esac
}

QA Agent Commands

New Lightweight Commands

*quick-audit           # Lightweight reality check (300-500 tokens)
*quick-audit --fix     # Include fix suggestions (500-700 tokens) 
*reality-audit         # Full comprehensive audit (3000+ tokens)
*reality-audit --full  # Force comprehensive audit

Token Usage Comparison

Audit Type	Token Cost	Use Case	Success Rate
Quick Audit	300-500	Routine checks	80% pass
Quick + Fixes	500-700	Minor issues	70% sufficient
Comprehensive	3,000+	Complex issues	100% coverage
Smart Hybrid	500-3,500	Adaptive	85% optimal

Expected Token Savings

Scenario Analysis

10 stories/day:
- Old: 10 × 3,000 = 30,000 tokens
- New: 8 × 500 + 2 × 3,000 = 10,000 tokens
- Savings: 67%
Simple stories (80% of cases):
- Old: 3,000 tokens each
- New: 500 tokens each
- Savings: 83%

Success Criteria

Quick simulation detection (300-500 tokens)
Accurate pass/fail decisions (80%+ accuracy)
Smart escalation to comprehensive audit
60-80% token savings for routine audits
Integration with story completion workflow
Maintain quality standards while reducing cost

This provides massive efficiency gains while preserving the comprehensive audit capability when truly needed!

5.9 KiB Raw Blame History Unescape Escape