694 lines
19 KiB
Markdown
694 lines
19 KiB
Markdown
# BMAD-SPEC-KIT Optimization Analysis
|
|
## Deep Dive Investigation - Session 011CV55cfUukw8yqP9kAYs58
|
|
|
|
### Executive Summary
|
|
|
|
This document identifies critical gaps between the documented capabilities of BMAD-SPEC-KIT and its actual implementation, along with optimization opportunities based on modern Claude Code agent-based tools and capabilities.
|
|
|
|
---
|
|
|
|
## Critical Findings
|
|
|
|
### 1. **Documentation vs Implementation Gap** (CRITICAL)
|
|
|
|
**Issue**: Sophisticated features are documented but not implemented in actual workflows.
|
|
|
|
**Evidence**:
|
|
- `parallel-execution-engine.md` describes Group 3 parallel execution (ux_expert + architect)
|
|
- `greenfield-fullstack.yaml` workflow runs all steps sequentially despite dependencies allowing parallelism
|
|
- No `parallel_groups` field exists in workflow YAML files
|
|
- Context engine describes automatic context injection, but workflows use manual file operations
|
|
|
|
**Impact**: System runs significantly slower than designed capability (40-60% performance loss)
|
|
|
|
**Solution**: Implement actual parallel execution in workflow files and orchestrator
|
|
|
|
---
|
|
|
|
### 2. **Manual Tool Orchestration** (HIGH PRIORITY)
|
|
|
|
**Issue**: Agents manually execute CLI tools instead of automated orchestration.
|
|
|
|
**Evidence** from `bmad-orchestrator/prompt.md`:
|
|
```yaml
|
|
Tooling steps (follow Context Protocol):
|
|
- Write route decision to `.claude/context/artifacts/route-decision.json`.
|
|
- Gate it: `node .claude/tools/gates/gate.mjs --schema ... --autofix 1`.
|
|
- On pass, embed the object into `.claude/context/session.json`
|
|
```
|
|
|
|
**Problems**:
|
|
- Error-prone manual command execution
|
|
- No unified error handling
|
|
- Difficult to maintain
|
|
- No transaction semantics (partial failures leave inconsistent state)
|
|
|
|
**Solution**: Create unified orchestration API that automates validate → render → update pipeline
|
|
|
|
---
|
|
|
|
### 3. **Missing Modern Task Tool Patterns** (HIGH PRIORITY)
|
|
|
|
**Issue**: System doesn't use Claude Code's Task tool for agent spawning.
|
|
|
|
**Current**: Agents are "role-played" by main Claude instance with prompt switching
|
|
**Modern**: Use Task tool with subagent_type to spawn actual specialized agents
|
|
|
|
**Benefits of Task Tool Pattern**:
|
|
- True parallel execution with concurrent agents
|
|
- Isolated context per agent
|
|
- Better error recovery
|
|
- Resource optimization
|
|
- Cleaner state management
|
|
|
|
**Solution**: Rewrite orchestrator to use Task tool for agent invocation
|
|
|
|
---
|
|
|
|
### 4. **Context Management Inefficiency** (MEDIUM PRIORITY)
|
|
|
|
**Issue**: File-based context with manual updates instead of structured data bus.
|
|
|
|
**Current Pattern**:
|
|
```json
|
|
{
|
|
"agent_contexts": {
|
|
"analyst": {
|
|
"outputs": {
|
|
"project_brief": {
|
|
"file_reference": "artifacts/project-brief.md",
|
|
"structured_data": { ... }
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Problems**:
|
|
- File I/O overhead on every access
|
|
- No type safety
|
|
- Manual context propagation
|
|
- Risk of stale data
|
|
- No transactional updates
|
|
|
|
**Solution**: Implement in-memory context store with schema validation and automatic propagation
|
|
|
|
---
|
|
|
|
### 5. **Agent Redundancy** (MEDIUM PRIORITY)
|
|
|
|
**Issue**: Overlapping responsibilities between agents.
|
|
|
|
**Redundancies Identified**:
|
|
1. **PM vs Product Owner** - Both handle requirements and prioritization
|
|
2. **Scrum Master vs PM** - Both manage workflows
|
|
3. **BMAD Master vs BMAD Orchestrator** - Unclear separation
|
|
|
|
**Impact**: Confusion about which agent to use, potential conflicts
|
|
|
|
**Solution**: Consolidate to core agents: Analyst, PM, Architect, Developer, QA, UX Expert
|
|
|
|
---
|
|
|
|
### 6. **No Feedback Loops** (HIGH PRIORITY)
|
|
|
|
**Issue**: One-directional workflow (Analysis → Design → Implementation) with no backpropagation.
|
|
|
|
**Missing Patterns**:
|
|
- Architect findings don't re-validate PM requirements
|
|
- QA failures don't trigger requirement clarification
|
|
- Developer implementation constraints don't update architecture
|
|
|
|
**Example Scenario**:
|
|
1. PM defines requirement: "Real-time collaboration"
|
|
2. Architect designs WebSocket architecture
|
|
3. Developer discovers hosting doesn't support WebSockets
|
|
4. **NO MECHANISM** to propagate this back to PM to revise requirements
|
|
|
|
**Solution**: Implement validation callbacks and adaptive re-routing
|
|
|
|
---
|
|
|
|
### 7. **Incomplete Schema Coverage** (MEDIUM PRIORITY)
|
|
|
|
**Current Schemas** (12 total):
|
|
- ✅ Artifact schemas (PRD, architecture, etc.)
|
|
- ❌ Process schemas (execution trace, quality metrics)
|
|
- ❌ Context state schema
|
|
- ❌ Error/recovery schemas
|
|
|
|
**Missing**:
|
|
- `execution_trace.schema.json` - Complete audit log
|
|
- `quality_metrics.schema.json` - Aggregated quality scores
|
|
- `context_state.schema.json` - Full context structure validation
|
|
- `error_report.schema.json` - Structured error reporting
|
|
|
|
**Solution**: Add process and state schemas for complete validation coverage
|
|
|
|
---
|
|
|
|
### 8. **Workflow YAML Limitations** (MEDIUM PRIORITY)
|
|
|
|
**Current Workflow Structure**:
|
|
```yaml
|
|
sequence:
|
|
- step: 3
|
|
agent: ux-expert
|
|
depends_on: [2]
|
|
- step: 5
|
|
agent: architect
|
|
depends_on: [2, 3] # Depends on UX, but could run parallel with it
|
|
```
|
|
|
|
**Problems**:
|
|
- No explicit parallel execution support
|
|
- Dependencies expressed but not optimized
|
|
- No conditional step execution
|
|
- No dynamic workflow adaptation
|
|
|
|
**Solution**: Enhanced workflow format with parallel groups and conditional execution
|
|
|
|
---
|
|
|
|
## Optimization Opportunities
|
|
|
|
### Priority 1: Core Performance & Architecture
|
|
|
|
#### A. Implement True Parallel Execution
|
|
|
|
**Enhanced Workflow Format**:
|
|
```yaml
|
|
workflow:
|
|
name: greenfield-fullstack-v2
|
|
execution_engine: parallel_optimized
|
|
|
|
parallel_groups:
|
|
- group_id: foundation
|
|
parallel: false
|
|
agents:
|
|
- { step: 1, agent: analyst }
|
|
- { step: 2, agent: pm, depends_on: [1] }
|
|
|
|
- group_id: design_and_architecture
|
|
parallel: true # KEY OPTIMIZATION
|
|
sync_barrier: smart_barrier
|
|
timeout: 10m
|
|
agents:
|
|
- { step: 3, agent: ux-expert, depends_on: [2] }
|
|
- { step: 4, agent: architect, depends_on: [2] }
|
|
|
|
- group_id: implementation
|
|
parallel: false
|
|
agents:
|
|
- { step: 5, agent: developer, depends_on: [3, 4] }
|
|
- { step: 6, agent: qa, depends_on: [5] }
|
|
```
|
|
|
|
**Implementation**:
|
|
1. Parse workflow YAML to identify parallel groups
|
|
2. Use Claude Code Task tool to spawn concurrent agents
|
|
3. Implement smart barrier synchronization
|
|
4. Handle partial completion scenarios
|
|
|
|
---
|
|
|
|
#### B. Modern Task Tool Integration
|
|
|
|
**Current Agent Invocation**:
|
|
```
|
|
As the ANALYST agent, I will now...
|
|
[Load analyst prompt and execute]
|
|
```
|
|
|
|
**Optimized Pattern**:
|
|
```javascript
|
|
// Orchestrator spawns agents using Task tool
|
|
invoke Task {
|
|
subagent_type: "general-purpose",
|
|
description: "Analyst agent: Project analysis",
|
|
prompt: `
|
|
You are the Analyst agent for BMAD-SPEC-KIT.
|
|
|
|
Load agent definition from: .claude/agents/analyst/prompt.md
|
|
Load enterprise rules from: .claude/rules/writing.md
|
|
|
|
Input Context:
|
|
${JSON.stringify(sessionContext.inputs.user_spec)}
|
|
|
|
Output Requirements:
|
|
- Generate project-brief.json conforming to .claude/schemas/project_brief.schema.json
|
|
- Run validation: node .claude/tools/gates/gate.mjs ...
|
|
- Return structured output with quality metrics
|
|
|
|
Return ONLY the validated JSON output.
|
|
`
|
|
}
|
|
```
|
|
|
|
**Benefits**:
|
|
- True concurrent execution
|
|
- Isolated agent context
|
|
- Automatic error handling
|
|
- Better resource management
|
|
|
|
---
|
|
|
|
#### C. Unified Orchestration API
|
|
|
|
**Create**: `.claude/tools/orchestrator/execute-step.mjs`
|
|
|
|
```javascript
|
|
#!/usr/bin/env node
|
|
/**
|
|
* Unified Step Executor
|
|
* Automates: validate → render → update pipeline
|
|
*/
|
|
|
|
import { validateWithGate } from './gates/gate.mjs';
|
|
import { renderArtifact } from './renderers/bmad-render.mjs';
|
|
import { updateSession } from './context/update-session.mjs';
|
|
|
|
async function executeStep(stepConfig, sessionContext) {
|
|
const { agent, schema, input, output, renderer } = stepConfig;
|
|
|
|
try {
|
|
// 1. Execute agent (via Task tool or direct)
|
|
const agentOutput = await executeAgent(agent, input, sessionContext);
|
|
|
|
// 2. Validate output
|
|
const validation = await validateWithGate({
|
|
schema,
|
|
input: agentOutput,
|
|
autofix: true
|
|
});
|
|
|
|
if (!validation.passed) {
|
|
throw new Error(`Validation failed: ${validation.errors}`);
|
|
}
|
|
|
|
// 3. Render artifact
|
|
const markdown = await renderArtifact(renderer, agentOutput);
|
|
|
|
// 4. Update session context (transactional)
|
|
await updateSession({
|
|
agent,
|
|
output: agentOutput,
|
|
rendered: markdown,
|
|
validation
|
|
});
|
|
|
|
return { success: true, output: agentOutput };
|
|
|
|
} catch (error) {
|
|
// Unified error handling with recovery options
|
|
return handleStepError(error, stepConfig, sessionContext);
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### Priority 2: Context Management
|
|
|
|
#### D. Structured Context Bus
|
|
|
|
**Create**: `.claude/tools/context/context-bus.mjs`
|
|
|
|
```javascript
|
|
/**
|
|
* In-Memory Context Bus with Schema Validation
|
|
* Replaces file-based context with structured data store
|
|
*/
|
|
|
|
class ContextBus {
|
|
constructor(sessionSchema) {
|
|
this.context = this.initializeFromSchema(sessionSchema);
|
|
this.subscribers = new Map(); // For reactive updates
|
|
this.history = []; // For time-travel debugging
|
|
}
|
|
|
|
// Type-safe context access
|
|
get(path) {
|
|
return _.get(this.context, path);
|
|
}
|
|
|
|
// Validated context updates
|
|
set(path, value, schema) {
|
|
// 1. Validate against schema
|
|
if (!this.validate(value, schema)) {
|
|
throw new ContextValidationError(path, value, schema);
|
|
}
|
|
|
|
// 2. Update with transaction semantics
|
|
const oldValue = this.get(path);
|
|
_.set(this.context, path, value);
|
|
|
|
// 3. Notify subscribers (reactive updates)
|
|
this.notifySubscribers(path, value, oldValue);
|
|
|
|
// 4. Record history
|
|
this.history.push({ path, value, oldValue, timestamp: Date.now() });
|
|
}
|
|
|
|
// Cross-agent data propagation
|
|
propagate(sourceAgent, targetAgent, mapping) {
|
|
const sourceData = this.get(`agents.${sourceAgent}.outputs`);
|
|
const transformedData = this.transform(sourceData, mapping);
|
|
this.set(`agents.${targetAgent}.inputs`, transformedData);
|
|
}
|
|
|
|
// Checkpoint/restore for error recovery
|
|
checkpoint() {
|
|
return JSON.parse(JSON.stringify(this.context));
|
|
}
|
|
|
|
restore(checkpoint) {
|
|
this.context = checkpoint;
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
#### E. Automatic Context Injection
|
|
|
|
**Enhance Agent Prompts**:
|
|
|
|
```markdown
|
|
# Agent Prompt Template (Enhanced)
|
|
|
|
## Context Injection (Automatic)
|
|
<!-- This section is automatically populated by the orchestrator -->
|
|
|
|
### Available Context
|
|
{{CONTEXT_INJECTION_START}}
|
|
<!-- Orchestrator injects structured context here -->
|
|
{
|
|
"from_analyst": {
|
|
"problem_statement": "...",
|
|
"target_users": ["..."],
|
|
"complexity_score": 7.2
|
|
},
|
|
"from_pm": {
|
|
"functional_requirements": [...],
|
|
"user_stories_count": 15
|
|
},
|
|
"global_context": {
|
|
"budget_level": "startup",
|
|
"quality_threshold": 8.0
|
|
}
|
|
}
|
|
{{CONTEXT_INJECTION_END}}
|
|
|
|
### Context Validation
|
|
Required inputs:
|
|
{{#each required_inputs}}
|
|
- [x] {{this.name}}: {{this.status}}
|
|
{{/each}}
|
|
|
|
## Agent Instructions
|
|
[Rest of agent prompt...]
|
|
```
|
|
|
|
---
|
|
|
|
### Priority 3: Quality & Reliability
|
|
|
|
#### F. Feedback Loop System
|
|
|
|
**Create**: `.claude/orchestrator/feedback-loop-engine.md`
|
|
|
|
```yaml
|
|
feedback_mechanisms:
|
|
validation_failure_callbacks:
|
|
architect_validates_pm:
|
|
trigger: "architect finds technical infeasibility"
|
|
action:
|
|
- notify_pm: "requirement X is not technically feasible"
|
|
- request_revision: true
|
|
- preserve_context: "architect's analysis"
|
|
- re_execute: ["pm.step_2"]
|
|
|
|
qa_validates_implementation:
|
|
trigger: "qa finds missing functionality"
|
|
action:
|
|
- trace_to_requirement: true
|
|
- identify_gap: "pm.requirements vs developer.implementation"
|
|
- escalate_to: ["pm", "developer"]
|
|
- decision: "add_requirement OR mark_as_limitation"
|
|
|
|
constraint_backpropagation:
|
|
developer_finds_constraint:
|
|
trigger: "developer identifies implementation limitation"
|
|
examples:
|
|
- "hosting doesn't support WebSockets"
|
|
- "database performance insufficient for requirement"
|
|
action:
|
|
- notify_architect: "constraint X affects architecture"
|
|
- architect_revises: "system_architecture"
|
|
- notify_pm: "may affect requirements"
|
|
- pm_decides: "revise_requirement OR change_hosting"
|
|
```
|
|
|
|
**Implementation Pattern**:
|
|
```javascript
|
|
// During developer agent execution
|
|
if (implementationConstraintFound) {
|
|
await feedbackLoop.notify({
|
|
source: "developer",
|
|
target: ["architect", "pm"],
|
|
issue: {
|
|
type: "constraint_violation",
|
|
requirement_id: "REQ-123",
|
|
constraint: "WebSocket not supported",
|
|
severity: "blocking"
|
|
},
|
|
requestedAction: "architecture_revision"
|
|
});
|
|
|
|
// Pause workflow pending resolution
|
|
await workflow.pause();
|
|
|
|
// Wait for architect to revise
|
|
const revision = await waitForRevision("architect", "system_architecture");
|
|
|
|
// Resume with updated context
|
|
await workflow.resume(revision);
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
#### G. Enhanced Schema Coverage
|
|
|
|
**Create Missing Schemas**:
|
|
|
|
1. **`execution_trace.schema.json`**:
|
|
```json
|
|
{
|
|
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
"type": "object",
|
|
"properties": {
|
|
"session_id": { "type": "string" },
|
|
"workflow_name": { "type": "string" },
|
|
"execution_log": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "object",
|
|
"properties": {
|
|
"timestamp": { "type": "string", "format": "date-time" },
|
|
"step_id": { "type": "integer" },
|
|
"agent": { "type": "string" },
|
|
"action": { "type": "string" },
|
|
"status": { "enum": ["started", "completed", "failed", "retrying"] },
|
|
"duration_ms": { "type": "integer" },
|
|
"quality_score": { "type": "number" },
|
|
"errors": { "type": "array" }
|
|
},
|
|
"required": ["timestamp", "step_id", "agent", "action", "status"]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
2. **`quality_metrics.schema.json`**:
|
|
```json
|
|
{
|
|
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
"type": "object",
|
|
"properties": {
|
|
"session_id": { "type": "string" },
|
|
"overall_quality_score": { "type": "number", "minimum": 0, "maximum": 10 },
|
|
"agent_scores": {
|
|
"type": "object",
|
|
"patternProperties": {
|
|
"^[a-z_]+$": {
|
|
"type": "object",
|
|
"properties": {
|
|
"completeness": { "type": "number" },
|
|
"clarity": { "type": "number" },
|
|
"technical_quality": { "type": "number" },
|
|
"overall": { "type": "number" }
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"validation_results": {
|
|
"type": "object",
|
|
"properties": {
|
|
"total_validations": { "type": "integer" },
|
|
"passed": { "type": "integer" },
|
|
"failed": { "type": "integer" },
|
|
"auto_fixed": { "type": "integer" }
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### Priority 4: Developer Experience
|
|
|
|
#### H. Agent Consolidation
|
|
|
|
**Consolidate Redundant Agents**:
|
|
|
|
**Before** (10 agents):
|
|
- analyst, pm, product-owner, scrum-master, architect, developer, qa, ux-expert, bmad-master, bmad-orchestrator
|
|
|
|
**After** (7 agents):
|
|
- **analyst** - Requirements gathering & analysis
|
|
- **pm** - Product requirements & user stories (absorbs product-owner, scrum-master)
|
|
- **architect** - System design & technology decisions
|
|
- **developer** - Implementation & code generation
|
|
- **qa** - Testing & quality assurance
|
|
- **ux-expert** - UI/UX design & accessibility
|
|
- **orchestrator** - Workflow coordination (absorbs bmad-master)
|
|
|
|
**Benefits**:
|
|
- Clearer responsibilities
|
|
- Fewer handoffs
|
|
- Reduced coordination overhead
|
|
- Simpler mental model
|
|
|
|
---
|
|
|
|
#### I. Simplified Workflow Activation
|
|
|
|
**Current Activation** (complex semantic analysis):
|
|
```yaml
|
|
activation_triggers:
|
|
- semantic_analysis: "understands creation intent"
|
|
- confidence_scoring: "calculates likelihood"
|
|
- context_extraction: "identifies project type"
|
|
```
|
|
|
|
**Optimized Activation**:
|
|
```yaml
|
|
simple_activation:
|
|
patterns:
|
|
- "create {project_type}"
|
|
- "build {description}"
|
|
- "implement {feature}"
|
|
|
|
clarification_prompts:
|
|
- "What type of project? (web-app | service | ui-only)"
|
|
- "Greenfield or brownfield?"
|
|
- "Any specific requirements or constraints?"
|
|
|
|
decision_tree:
|
|
- if project_type == "web-app" and greenfield: use greenfield-fullstack
|
|
- if project_type == "service": use greenfield-service
|
|
- if brownfield: ask about existing codebase
|
|
```
|
|
|
|
---
|
|
|
|
## Implementation Roadmap
|
|
|
|
### Phase 1: Core Architecture (Week 1)
|
|
1. ✅ Complete optimization analysis
|
|
2. Implement parallel execution in workflow YAML
|
|
3. Create unified orchestration API
|
|
4. Add missing schemas
|
|
|
|
### Phase 2: Modern Patterns (Week 2)
|
|
1. Integrate Task tool for agent spawning
|
|
2. Implement context bus
|
|
3. Add automatic context injection
|
|
4. Test parallel execution
|
|
|
|
### Phase 3: Quality & Reliability (Week 3)
|
|
1. Implement feedback loop system
|
|
2. Add comprehensive error recovery
|
|
3. Create execution trace logging
|
|
4. Performance benchmarking
|
|
|
|
### Phase 4: Refinement (Week 4)
|
|
1. Consolidate redundant agents
|
|
2. Update documentation
|
|
3. Create migration guide
|
|
4. User acceptance testing
|
|
|
|
---
|
|
|
|
## Expected Impact
|
|
|
|
### Performance Improvements
|
|
- **40-60% faster execution** via parallel agent execution
|
|
- **30% reduction** in manual operations
|
|
- **50% faster error recovery** via unified orchestration
|
|
|
|
### Quality Improvements
|
|
- **Zero manual tool invocation errors** via automation
|
|
- **100% schema coverage** for all artifacts and processes
|
|
- **Real-time quality feedback** via context bus
|
|
|
|
### Developer Experience
|
|
- **Simpler agent model** (7 vs 10 agents)
|
|
- **Clearer responsibilities** via consolidation
|
|
- **Better error messages** via structured error schemas
|
|
- **Easier debugging** via execution traces
|
|
|
|
---
|
|
|
|
## Risk Assessment
|
|
|
|
### Low Risk
|
|
- Adding missing schemas (backward compatible)
|
|
- Documentation updates
|
|
- Agent consolidation (optional migration path)
|
|
|
|
### Medium Risk
|
|
- Context bus implementation (need migration from file-based)
|
|
- Feedback loop system (new workflow capability)
|
|
|
|
### High Risk (Mitigation Required)
|
|
- Task tool integration (fundamental change to execution model)
|
|
- **Mitigation**: Implement feature flag, run both modes in parallel
|
|
- Parallel execution engine (complex synchronization)
|
|
- **Mitigation**: Start with simple 2-agent parallel, expand gradually
|
|
|
|
---
|
|
|
|
## Conclusion
|
|
|
|
BMAD-SPEC-KIT has excellent architectural design in its documentation but significant gaps in implementation. The optimizations outlined above will:
|
|
|
|
1. Bridge the documentation-implementation gap
|
|
2. Leverage modern Claude Code capabilities
|
|
3. Deliver 40-60% performance improvements
|
|
4. Improve reliability and developer experience
|
|
5. Maintain backward compatibility where possible
|
|
|
|
**Recommendation**: Proceed with implementation in phases, starting with Phase 1 (Core Architecture) which delivers immediate value with low risk.
|
|
|
|
---
|
|
|
|
**Document Version**: 1.0
|
|
**Date**: 2025-11-13
|
|
**Session**: claude/deep-dive-investigation-011CV55cfUukw8yqP9kAYs58
|