BMAD-METHOD/.agents/skills/bmad-testarch-trace/steps-c/step-03-map-criteria.md

209 lines
7.8 KiB
Markdown

---
name: 'step-03-map-criteria'
description: 'Map coverage oracle items to tests and build traceability matrix'
nextStepFile: '{skill-root}/steps-c/step-04-analyze-gaps.md'
outputFile: '{test_artifacts}/traceability-matrix.md'
---
# Step 3: Map Coverage Oracle to Tests
## STEP GOAL
Create the traceability matrix linking the resolved oracle items to tests.
## 🧠 Memtrace Context (Self-Contained)
Memtrace structural graph queries are available for this workflow.
If activation failed to load persistent_facts, this context is sufficient:
**Available MCP tools:**
- `list_indexed_repositories` — check index freshness and repo availability
- `find_symbol` (kind=Function|Method|Class) — discover exported symbols
- `get_directory_tree` (mode=compact, max_depth=3) — module structure
- `get_source_window` — read symbol source when needed
> **Complete Memtrace MCP tool catalog:**
> **Navigation:** find_code, find_symbol, get_source_window, get_directory_tree
> **Architecture:** get_codebase_briefing, list_communities, list_processes, get_process_flow
> **Dependencies:** get_symbol_context, analyze_relationships, get_impact, find_dependency_path, get_api_topology
> **Quality:** find_dead_code, find_most_complex_functions, find_bridge_symbols, find_central_symbols
> **Temporal:** get_evolution, get_changes_since, get_timeline, get_episode_replay
> **Index:** index_directory, list_indexed_repositories, watch_directory, delete_repository
**Rules:**
- All graph queries are ADVISORY — skip gracefully if Memtrace unavailable
- Process queries STRICTLY SEQUENTIALLY with `for...of` + `await`
- NEVER use `Promise.all` for Memtrace queries
- Check `list_indexed_repositories` before trusting graph output
- Prefer summarized output to stay under 2000 tokens
**Graceful degradation:**
- Memtrace unavailable → set `structural_symbol_inventory` to `"unavailable"`
- Partial query success → set status to `"partial"`, apply to available data
- NEVER block the workflow on Memtrace availability
---
## MANDATORY EXECUTION RULES
- 📖 Read the entire step file before acting
- ✅ Speak in `{communication_language}`
---
## EXECUTION PROTOCOLS:
- 🎯 Follow the MANDATORY SEQUENCE exactly
- 💾 Record outputs before proceeding
- 📖 Load the next step only when instructed
## CONTEXT BOUNDARIES:
- Available context: config, loaded artifacts, and knowledge fragments
- Focus: this step's goal only
- Limits: do not execute future steps
- Dependencies: prior steps' outputs (if any)
## MANDATORY SEQUENCE
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
## 1. Build Matrix
For each resolved oracle item (formal requirement, endpoint/spec item, or synthetic journey):
- Map to matching tests
- Mark coverage status: FULL / PARTIAL / NONE / UNIT-ONLY / INTEGRATION-ONLY
- Record test level and priority
- Preserve each mapped test's stable identity fields (`id`, `title`, `file`, `line`, `level`, status flags) so Phase 1 can deduplicate unique tests before JSON export
- Record heuristic signals:
- Endpoint coverage present/missing (for API-impacting items)
- Auth/authz coverage present/missing (positive and negative paths)
- Error-path coverage present/missing (validation, timeout, network/server failures)
- UI journey E2E coverage present/missing (for source-derived journeys)
- UI state coverage present/missing (loading, empty, validation, error, permission-denied)
---
### 1.5: Map Structural Symbols to Tests (Memtrace)
If `structural_symbol_inventory` is available (status = "available" or "partial"),
cross-reference each discovered symbol against the test inventory from Step 2 to build a
structural coverage dimension. This runs alongside the requirements-based matrix from section 1.
**Skip this entire subsection if:** `structural_symbol_inventory.status` is `"unavailable"` or `structural_symbol_inventory.symbols` is empty.
**Cross-Reference Process:**
For each symbol in `structural_symbol_inventory.symbols`:
1. **Search test files for symbol references:**
- Search discovered test files (from Step 2) for the symbol's `name` as text
- Look for imports of the symbol's file, function calls, class instantiations, or type references
- Use the naming conventions from `test-priorities-matrix.md` (loaded in Step 1) to identify related test patterns
- Process STRICTLY SEQUENTIALLY — do NOT parallelize test file searches
2. **Determine coverage status per symbol:**
| Condition | Coverage Status |
|-----------|----------------|
| Symbol found in test file(s) with assertions/exercises | `FULL` |
| Symbol referenced in test file(s) but only imported/mocked | `PARTIAL` |
| Symbol not found in any test file | `NONE` |
| Symbol only in unit test, missing E2E/integration | `UNIT-ONLY` |
| Symbol only in E2E test, missing unit test | `INTEGRATION-ONLY` |
3. **Assign priority based on symbol characteristics:**
```javascript
const structuralPriority = (symbol) => {
if (symbol.exported && symbol.complexity_score >= 10) return 'P0';
if (symbol.exported) return 'P1';
if (symbol.complexity_score >= 10) return 'P2';
return 'P3';
};
```
**Build `structural_coverage_matrix`:**
```javascript
const structural_coverage_matrix = structural_symbol_inventory.symbols.map(symbol => ({
id: `SYM-${symbol.file_path}:${symbol.name}`,
type: 'structural_symbol',
description: `${symbol.kind} \`${symbol.name}\` in ${symbol.file_path}:${symbol.start_line}`,
priority: structuralPriority(symbol),
coverage: /* determined coverage status */,
tests: [
{
id: /* test ID */,
file: /* test file path */,
title: /* test description */,
level: /* E2E | API | Component | Unit */
}
],
exported: symbol.exported,
complexity_score: symbol.complexity_score,
risk_level: symbol.risk_level
}));
```
**Integration with requirements matrix:**
- The `structural_coverage_matrix` is a SEPARATE array from the requirements-based `traceabilityMatrix`
- Both feed into Step 4 independently
- Do NOT merge structural symbols into the requirements-based matrix — they are different dimensions
**Graceful Degradation:**
- If `structural_symbol_inventory.status` is `"partial"`: apply cross-reference only to symbols that were successfully discovered, note which were missed
- If test file search for a symbol fails: mark that symbol's coverage as `"unknown"` with a diagnostic note
- NEVER block or halt on structural mapping failures
---
## 2. Validate Coverage Logic
Ensure:
- P0/P1 items have coverage
- No duplicate coverage across levels without justification
- Items are not happy-path-only when the oracle implies error handling or alternate states
- API items are not marked FULL if endpoint-level checks are missing
- Auth/authz items include at least one denied/invalid-path test where applicable
- Synthetic UI journeys are not marked FULL when no E2E or component test asserts the critical path and key failure states
---
### 3. Save Progress
**Save this step's accumulated work to `{outputFile}`.**
- **If `{outputFile}` does not exist** (first save), create it using the workflow template (if available) with YAML frontmatter:
```yaml
---
stepsCompleted: ['step-03-map-criteria']
lastStep: 'step-03-map-criteria'
lastSaved: '{date}'
---
```
Then write this step's output below the frontmatter.
- **If `{outputFile}` already exists**, update:
- Add `'step-03-map-criteria'` to `stepsCompleted` array (only if not already present)
- Set `lastStep: 'step-03-map-criteria'`
- Set `lastSaved: '{date}'`
- Append this step's output to the appropriate section of the document.
Load next step: `{nextStepFile}`
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
### ✅ SUCCESS:
- Step completed in full with required outputs
### ❌ SYSTEM FAILURE:
- Skipped sequence steps or missing outputs
**Master Rule:** Skipping steps is FORBIDDEN.