226 lines
8.9 KiB
Markdown
226 lines
8.9 KiB
Markdown
<!-- Powered by BMAD™ Core -->
|
|
|
|
# Shard-Aware Evaluation Task
|
|
|
|
## Purpose
|
|
|
|
- Enable comprehensive evaluation of sharded proposal content
|
|
- Implement cross-shard analysis capabilities for SWARD analysis
|
|
- Add metadata to evaluation results linking back to specific shards
|
|
- Create consolidated evaluation views that span multiple shards
|
|
- Improve evaluation efficiency and depth across large proposal content
|
|
|
|
## Usage Scenarios
|
|
|
|
### Scenario 1: Sharded Technical Volume Evaluation
|
|
|
|
1. **Shard Inventory**: Identify all technical approach content shards
|
|
2. **Cross-Shard Review**: Evaluate technical content across shard boundaries
|
|
3. **Consolidated Analysis**: Aggregate evaluation findings from all technical shards
|
|
4. **Metadata Linkage**: Track evaluation findings back to source shards
|
|
5. **Unified Recommendations**: Generate integrated improvement recommendations
|
|
|
|
### Scenario 2: Multi-Volume Sharded Evaluation
|
|
|
|
1. **Cross-Volume Analysis**: Analyze related content across different sharded volumes
|
|
2. **Requirement Traceability**: Track requirement implementation across all shards
|
|
3. **Consistency Verification**: Ensure uniform messaging across proposal shards
|
|
4. **Gap Identification**: Find cross-shard gaps and inconsistencies
|
|
5. **Strategic Enhancement**: Provide comprehensive strategy improvements spanning all shards
|
|
|
|
## Task Instructions
|
|
|
|
### 1. Shard-Aware Evaluation Preparation
|
|
|
|
**Preparation Process**:
|
|
|
|
1. **Shard Inventory and Mapping**:
|
|
- Review shard manifests for all proposal content
|
|
- Create evaluation matrix linking shards to Section M criteria
|
|
- Map relationships between related shards
|
|
- Identify dependencies and cross-references between shards
|
|
|
|
2. **Cross-Shard Context Building**:
|
|
- Review shard metadata to understand context
|
|
- Identify thematic connections between shards
|
|
- Document content that spans multiple shards
|
|
- Create cross-reference map for related concepts
|
|
|
|
3. **Evaluation Framework Extension**:
|
|
- Adapt evaluation criteria for sharded content
|
|
- Establish consistent scoring approach across shards
|
|
- Create cross-shard evaluation worksheets
|
|
- Develop metadata tagging schema for findings
|
|
|
|
### 2. Cross-Shard Evaluation Process
|
|
|
|
**Evaluation Methodology**:
|
|
|
|
1. **Individual Shard Review**:
|
|
- Evaluate each shard on its own merits
|
|
- Score against applicable Section M criteria
|
|
- Document strengths, weaknesses, risks, and deficiencies
|
|
- Tag findings with shard-specific metadata
|
|
|
|
2. **Boundary Analysis**:
|
|
- Review content transitions between related shards
|
|
- Identify inconsistencies at shard boundaries
|
|
- Assess narrative flow across shard divisions
|
|
- Evaluate context preservation between shards
|
|
|
|
3. **Cross-Shard Requirements Verification**:
|
|
- Track requirement implementation across multiple shards
|
|
- Verify complete coverage of multi-part requirements
|
|
- Identify redundancies or conflicts in requirement responses
|
|
- Assess consistency of approach for related requirements
|
|
|
|
4. **Theme and Message Consistency**:
|
|
- Evaluate win theme implementation across all shards
|
|
- Verify consistent messaging on key points
|
|
- Identify terminology variations between shards
|
|
- Assess uniform presentation of corporate capabilities
|
|
|
|
### 3. Metadata-Enhanced Findings Documentation
|
|
|
|
**Documentation Approach**:
|
|
|
|
1. **Shard-Specific Findings**:
|
|
- Document findings with shard identifiers
|
|
- Include location references within shards
|
|
- Tag with requirement IDs from compliance matrix
|
|
- Link to related findings in other shards
|
|
|
|
2. **Cross-Shard Findings**:
|
|
- Document issues that span multiple shards
|
|
- Identify root causes of cross-shard inconsistencies
|
|
- Map finding impacts across the proposal
|
|
- Create relationship diagrams for complex issues
|
|
|
|
3. **Metadata Enrichment**:
|
|
- Apply consistent taxonomy to all findings
|
|
- Tag findings with evaluation criteria references
|
|
- Include impact ratings and priority indicators
|
|
- Add context notes for findings at shard boundaries
|
|
|
|
4. **Consolidated Finding Repository**:
|
|
- Create centralized database of all findings
|
|
- Implement filtering by shard, criteria, and priority
|
|
- Enable sorting and grouping of related findings
|
|
- Provide visualization of finding distribution
|
|
|
|
### 4. Cross-Shard SWARD Analysis
|
|
|
|
**Analysis Process**:
|
|
|
|
1. **Strength Aggregation**:
|
|
- Identify strengths that span multiple shards
|
|
- Evaluate cumulative impact of related strengths
|
|
- Highlight themes of excellence across shards
|
|
- Create strength heat map across all content
|
|
|
|
2. **Weakness Pattern Recognition**:
|
|
- Detect recurring weaknesses across shards
|
|
- Identify systemic issues affecting multiple sections
|
|
- Create weakness pattern analysis
|
|
- Develop root cause assessment
|
|
|
|
3. **Risk Dependency Analysis**:
|
|
- Map risks that affect multiple proposal areas
|
|
- Identify compound risks spanning shards
|
|
- Evaluate cascading risk implications
|
|
- Create risk network visualization
|
|
|
|
4. **Deficiency Impact Assessment**:
|
|
- Evaluate how deficiencies affect overall compliance
|
|
- Identify critical pathway deficiencies
|
|
- Create deficiency severity matrix
|
|
- Map deficiency correction dependencies
|
|
|
|
### 5. Consolidated Evaluation Views
|
|
|
|
**Consolidation Techniques**:
|
|
|
|
1. **Executive Dashboards**:
|
|
- Create high-level view of evaluation across all shards
|
|
- Provide scoring summaries by evaluation criteria
|
|
- Highlight critical findings requiring attention
|
|
- Present strategic recommendation overview
|
|
|
|
2. **Heat Map Visualizations**:
|
|
- Generate heat maps showing evaluation results across shards
|
|
- Visualize strength and weakness distributions
|
|
- Create color-coded requirement coverage maps
|
|
- Show theme implementation effectiveness
|
|
|
|
3. **Finding Relationship Maps**:
|
|
- Visualize connections between related findings
|
|
- Map finding dependencies and impacts
|
|
- Show clustering of issues by topic or area
|
|
- Create navigable relationship diagrams
|
|
|
|
4. **Cross-Reference Navigator**:
|
|
- Enable navigation between related findings across shards
|
|
- Provide context transitions between shards
|
|
- Support drilling down from summary to detail
|
|
- Create traceable paths through evaluation data
|
|
|
|
### 6. Integrated Improvement Recommendations
|
|
|
|
**Recommendation Approach**:
|
|
|
|
1. **Cross-Shard Enhancement Strategy**:
|
|
- Develop holistic improvement strategy
|
|
- Prioritize changes that affect multiple shards
|
|
- Create coordinated enhancement approach
|
|
- Design consistent implementation guidance
|
|
|
|
2. **Boundary Improvement Recommendations**:
|
|
- Provide specific guidance for shard transitions
|
|
- Recommend context enhancements at boundaries
|
|
- Suggest narrative flow improvements
|
|
- Develop terminology standardization
|
|
|
|
3. **Theme Reinforcement Recommendations**:
|
|
- Suggest consistent theme application across shards
|
|
- Recommend balanced theme emphasis
|
|
- Provide cross-shard messaging guidance
|
|
- Create theme implementation templates
|
|
|
|
4. **Implementation Coordination**:
|
|
- Develop synchronized revision approach
|
|
- Create cross-team coordination plan
|
|
- Establish consistency verification milestones
|
|
- Design final integration validation process
|
|
|
|
## Best Practices
|
|
|
|
- **Holistic Perspective**: Maintain awareness of the complete proposal context when evaluating individual shards
|
|
- **Boundary Focus**: Pay special attention to content transitions between shards
|
|
- **Metadata Discipline**: Consistently tag all findings with appropriate shard references and metadata
|
|
- **Cross-Reference Mapping**: Maintain clear relationships between findings across different shards
|
|
- **Context Preservation**: Ensure evaluation considers the intended context of each shard
|
|
- **Consistent Standards**: Apply uniform evaluation criteria across all shards
|
|
- **Integrated Visualization**: Use visual tools to represent findings across multiple shards
|
|
- **Coordinated Recommendations**: Ensure improvement recommendations consider impacts across all related shards
|
|
|
|
## Integration Points
|
|
|
|
- **RFQ Document Sharding**: Aligns with sharded RFQ documents for evaluation against requirements
|
|
- **Compliance Requirement Sharding**: Uses sharded requirements for compliance verification
|
|
- **Proposal Content Sharding**: Evaluates sharded proposal content with awareness of relationships
|
|
- **Shard Visualization Tools**: Feeds evaluation data to visualization systems
|
|
- **Smart Shard Reassembly**: Provides input for improvement implementation during reassembly
|
|
|
|
## Related Agents
|
|
|
|
- **Evaluator Simulator**: Primary agent for evaluation across sharded content
|
|
- **Compliance Matrix Builder**: Provider of requirement traceability data
|
|
- **Technical Narrative Drafter**: Consumer of shard-specific evaluation findings
|
|
|
|
## Technical Implementation Notes
|
|
|
|
- Extends the existing review simulation task with shard-aware capabilities
|
|
- Compatible with the sharded content development workflow
|
|
- Enables comprehensive evaluation across arbitrary shard boundaries
|
|
- Maintains evaluation consistency regardless of content sharding approach
|