Brian Madison
|
48cf5c8056
|
feat(workflows): Implement intelligent file discovery protocol and Phase 4 BMGD workflows
## Core Workflow Engine Enhancements
### discover_inputs Protocol (MAJOR)
- Added reusable `discover_inputs` protocol to workflow.xml for intelligent file loading
- Supports three loading strategies:
- FULL_LOAD: Load all shards for PRD, Architecture, UX (changed pattern from /index.md to /*/*.md)
- SELECTIVE_LOAD: Load specific shard via template variable (e.g., epic-{{epic_num}}.md)
- INDEX_GUIDED: Load index, analyze TOC, intelligently load relevant docs (with "DO NOT BE LAZY" mandate)
- Auto-discovers whole vs sharded documents with proper fallback
- Provides transparent reporting of loaded content with file counts
- Invoked via <invoke-protocol name="discover_inputs" /> tag in workflow instructions
### Advanced Elicitation Improvements
- Renamed adv-elicit.xml to advanced-elicitation.xml for clarity
- Updated all references across agents and commands
### Shard Document Tool Enhancement
- Added Step 6: Handle Original Document with three options:
- [d] Delete - Remove original (recommended, prevents confusion)
- [m] Move to archive - Backup original to archive folder
- [k] Keep - Warning about defeating sharding purpose
- Prevents issue where both whole and sharded versions exist, confusing discover_inputs protocol
## BMM Module - Input File Pattern Standardization
### Phase 1 - Analysis (1 workflow)
- product-brief: Added load_strategy (FULL_LOAD for research/brainstorming, INDEX_GUIDED for document_project)
- Updated instructions.md to use invoke-protocol, replaced manual fuzzy matching
### Phase 2 - Planning (4 workflows)
- prd: Added load_strategy, updated instructions to reference {product_brief_content}, {research_content}
- create-ux-design: Added load_strategy, removed fuzzy matching from instructions
- tech-spec: Added load_strategy for brownfield context discovery
- All epics patterns updated to support SELECTIVE_LOAD for specific epic shards
### Phase 3 - Solutioning (2 workflows)
- architecture: Added load_strategy, updated instructions to use pre-loaded {prd_content}, {epics_content}, {ux_design_content}
- solutioning-gate-check: Added load_strategy, replaced manual discovery with protocol invocation
### Phase 4 - Implementation (8 workflows)
- code-review: Added load_strategy, fixed sharded patterns to /*/*.md, added step 1.5 for protocol
- correct-course: Added complete input_file_patterns section (was missing), added step 0.5
- create-story: Added load_strategy, updated to SELECTIVE_LOAD for epics, added step 1.5
- dev-story: Added complete input_file_patterns section (was missing), added step 0.5
- epic-tech-context: Added load_strategy, updated PRD extraction to use {prd_content}, added step 1.5
- retrospective: Added load_strategy for architecture/prd (FULL_LOAD), epics (SELECTIVE_LOAD), added step 0.5
- sprint-planning: Fixed sharded pattern to load ALL epics (/*/*.md), added step 0.5
- story-context: Added load_strategy, updated doc collection to reference pre-loaded content, added step 1.5
### Sprint Artifacts Path Corrections
- story-done: Added missing sprint_artifacts variable, fixed sprint_status path from {context_dir} to {sprint_artifacts}
- story-ready: Added missing sprint_artifacts variable
- story-context: Fixed undefined {context_dir} -> {sprint_artifacts}
- correct-course: Added sprint_artifacts and sprint_status variables
## BMGD Module - Phase 4 Production Workflows (NEW)
Added complete Phase 4 implementation workflows for game development:
- code-review: Senior developer review for completed game features
- correct-course: Sprint change management for game projects
- create-story: Story generation for game mechanics/features
- dev-story: Feature implementation workflow
- epic-tech-context: Technical spec generation per game epic
- retrospective: Epic completion review and lessons learned
- sprint-planning: Game development sprint status tracking
- story-context: Dynamic context assembly for game stories
- story-done: Story completion workflow
- story-ready: Story readiness workflow
All BMGD workflows follow BMM patterns with game-specific adaptations.
## Agent Updates
### BMM Agents
- Updated all 7 BMM agents (analyst, architect, dev, pm, sm, tea, tech-writer, ux-designer)
- Standardized web bundle configurations
### BMGD Agents
- Updated 4 game development agents (game-architect, game-designer, game-dev, game-scrum-master)
- Aligned with BMM agent structure
### CIS Agents
- Updated 5 creative intelligence agents for consistency
## Documentation & Configuration
- Updated CHANGELOG.md with Phase 4 workflow additions
- Updated files-manifest.csv and task-manifest.csv
- Updated .claude commands for all agents
- Fixed formatting issues from previous commits
## Breaking Changes
NONE - All changes are backward compatible. Workflows without input_file_patterns continue to work.
Workflows with input_file_patterns now benefit from intelligent auto-loading.
## Migration Notes
Existing workflows can gradually adopt discover_inputs protocol by:
1. Adding load_strategy to existing input_file_patterns in workflow.yaml
2. Adding <invoke-protocol name="discover_inputs" /> step in instructions.md
3. Replacing manual file loading with references to {pattern_name_content} variables
|
2025-11-12 19:18:38 -06:00 |
|
Brian Madison
|
3d4ea5ffd2
|
feat: add universal document sharding support with dual-strategy loading
Implement comprehensive document sharding system across all BMM workflows enabling 90%+ token savings for large multi-epic projects through selective loading optimization.
## Document Sharding System
### Core Features
- **Universal Support**: All 12 BMM workflows (Phase 1-4) handle both whole and sharded documents
- **Dual Loading Strategy**: Full Load (Phase 1-3) vs Selective Load (Phase 4)
- **Automatic Discovery**: Workflows detect format transparently (whole → sharded priority)
- **Efficiency Optimization**: 90%+ token reduction for 10+ epic projects in Phase 4
### Implementation Details
**Phase 1-3 Workflows (7 workflows) - Full Load Strategy:**
- product-brief, prd, gdd, create-ux-design, tech-spec, architecture, solutioning-gate-check
- Load entire sharded documents when present
- Transparent to user experience
- Better organization for large projects
**Phase 4 Workflows (5 workflows) - Selective Load Strategy:**
- sprint-planning (Full Load exception - needs all epics)
- epic-tech-context, create-story, story-context, code-review (Selective Load)
- Load ONLY the specific epic needed (e.g., epic-3.md for Epic 3 stories)
- Massive efficiency: Skip loading 9 other epics in 10-epic project
### Workflow Enhancements
**Added to all workflows:**
- `input_file_patterns` in workflow.yaml with wildcard discovery
- Document Discovery section in instructions.md
- Support for sharded index + section files
- Brownfield `docs/index.md` support
**Pattern standardization:**
```yaml
input_file_patterns:
document:
whole: "{output_folder}/*doc*.md"
sharded: "{output_folder}/*doc*/index.md"
sharded_single: "{output_folder}/*doc*/section-{{id}}.md" # Selective load
```
### Retrospective Workflow Major Overhaul
Transformed retrospective into immersive, interactive team experience:
**Epic Discovery Priority (Fixed):**
- Priority 1: Check sprint-status.yaml for last completed epic
- Priority 2: Ask user directly
- Priority 3: Scan stories folder (last resort)
**New Capabilities:**
- Deep story analysis: Extract dev notes, mistakes, review feedback, lessons learned
- Previous retro integration: Track action items, verify lessons applied
- Significant change detection: Alert when discoveries require epic updates
- Intent-based facilitation: Natural conversation vs scripted phrases
- Party mode protocol: Clear speaker identification (Name (Role): dialogue)
- Team dynamics: Drama, disagreements, diverse perspectives, authentic conflict
**Structure:**
- 12 whole-number steps (no decimals)
- Highly interactive with constant user engagement
- Cross-references previous retro for accountability
- Synthesizes patterns across all stories
- Detects architectural assumption changes
## Documentation
**Created:**
- `docs/document-sharding-guide.md` - Comprehensive 300+ line guide
- What is sharding, when to use it (token thresholds)
- How sharding works (discovery system, loading strategies)
- Using shard-doc tool
- Full Load vs Selective Load patterns
- Complete examples and troubleshooting
- Custom workflow integration patterns
**Updated:**
- `README.md` - Added Document Sharding feature section
- `docs/index.md` - Added under Advanced Topics → Optimization
- `src/modules/bmm/workflows/README.md` - Added sharding section with usage
- `src/modules/bmb/workflows/create-workflow/workflow-creation-guide.md` - Added complete implementation patterns for workflow builders
**Documentation levels:**
1. Overview (README.md) - Quick feature highlight
2. User guide (BMM workflows README) - Practical usage
3. Reference (document-sharding-guide.md) - Complete details
4. Builder guide (workflow-creation-guide.md) - Implementation patterns
## Efficiency Gains
**Example: 10-Epic Project**
Before sharding:
- epic-tech-context for Epic 3: Load all 10 epics (~50k tokens)
- create-story for Epic 3: Load all 10 epics (~50k tokens)
- story-context for Epic 3: Load all 10 epics (~50k tokens)
After sharding with selective load:
- epic-tech-context for Epic 3: Load Epic 3 only (~5k tokens) = 90% reduction
- create-story for Epic 3: Load Epic 3 only (~5k tokens) = 90% reduction
- story-context for Epic 3: Load Epic 3 only (~5k tokens) = 90% reduction
## Breaking Changes
None - fully backward compatible. Workflows work with existing whole documents.
## Files Changed
**Workflows Updated (25 files):**
- 7 Phase 1-3 workflows: Added full load sharding support
- 5 Phase 4 workflows: Added selective load sharding support
- 1 retrospective workflow: Complete overhaul with sharding support
**Documentation (5 files):**
- Created: document-sharding-guide.md
- Updated: README.md, docs/index.md, BMM workflows README, BMB workflow-creation-guide
- Removed: Old conversion report (obsolete)
## Future Extensibility
- BMB workflows now aware of sharding patterns
- Custom modules can easily implement sharding support
- Standard patterns documented for consistency
- No need to explain concept in future development
|
2025-11-02 00:13:33 -05:00 |