From 325a264412bebc25b8f60480c908022203b8c668 Mon Sep 17 00:00:00 2001 From: murat Date: Tue, 23 Dec 2025 08:57:49 -0600 Subject: [PATCH] docs: update test architecture documentation for clarity and consistency --- src/modules/bmm/docs/workflows-solutioning.md | 2 +- .../testarch/atdd/atdd-checklist-template.md | 15 ++++++++------- .../bmm/workflows/testarch/atdd/checklist.md | 1 + .../bmm/workflows/testarch/atdd/instructions.md | 1 + .../bmm/workflows/testarch/automate/checklist.md | 2 ++ .../bmm/workflows/testarch/ci/checklist.md | 2 ++ .../bmm/workflows/testarch/ci/instructions.md | 2 ++ .../workflows/testarch/nfr-assess/checklist.md | 2 ++ .../testarch/nfr-assess/nfr-report-template.md | 2 ++ .../workflows/testarch/test-design/checklist.md | 3 ++- .../testarch/test-design/instructions.md | 2 +- .../testarch/test-design/test-design-template.md | 9 +++++++++ .../workflows/testarch/test-review/checklist.md | 2 ++ .../testarch/test-review/test-review-template.md | 2 ++ .../bmm/workflows/testarch/trace/checklist.md | 1 + .../bmm/workflows/testarch/trace/instructions.md | 2 ++ .../workflows/testarch/trace/trace-template.md | 2 ++ 17 files changed, 42 insertions(+), 10 deletions(-) diff --git a/src/modules/bmm/docs/workflows-solutioning.md b/src/modules/bmm/docs/workflows-solutioning.md index 3ce4b5ac..9e682bd9 100644 --- a/src/modules/bmm/docs/workflows-solutioning.md +++ b/src/modules/bmm/docs/workflows-solutioning.md @@ -363,7 +363,7 @@ Planning (prd by PM - FRs/NFRs only) → Phase 4 (Implementation) ``` -**Note on TEA (Test Architect):** TEA is fully operational with 8 workflows across all phases. TEA validates architecture testability during Phase 3 reviews but does not have a dedicated solutioning workflow. TEA's primary setup occurs in Phase 2 (`*framework`, `*ci`, `*test-design`) and testing execution in Phase 4 (`*atdd`, `*automate`, `*test-review`, `*trace`, `*nfr-assess`). +**Note on TEA (Test Architect):** TEA is fully operational with 8 workflows across all phases. TEA validates architecture testability during Phase 3 reviews but does not have a dedicated solutioning workflow. TEA's primary setup occurs after architecture in Phase 3 (`*framework`, `*ci`, system-level `*test-design`), with optional Phase 2 baseline `*trace`. Testing execution happens in Phase 4 (`*atdd`, `*automate`, `*test-review`, `*trace`, `*nfr-assess`). **Note:** Enterprise uses the same planning and architecture as BMad Method. The only difference is optional extended workflows added AFTER architecture but BEFORE create-epics-and-stories. diff --git a/src/modules/bmm/workflows/testarch/atdd/atdd-checklist-template.md b/src/modules/bmm/workflows/testarch/atdd/atdd-checklist-template.md index 8eaa4f59..3a633c75 100644 --- a/src/modules/bmm/workflows/testarch/atdd/atdd-checklist-template.md +++ b/src/modules/bmm/workflows/testarch/atdd/atdd-checklist-template.md @@ -290,13 +290,14 @@ test('should do something', async ({ {fixtureName} }) => { ## Next Steps -1. **Review this checklist** with team in standup or planning -2. **Run failing tests** to confirm RED phase: `{test_command_all}` -3. **Begin implementation** using implementation checklist as guide -4. **Work one test at a time** (red → green for each) -5. **Share progress** in daily standup -6. **When all tests pass**, refactor code for quality -7. **When refactoring complete**, manually update story status to 'done' in sprint-status.yaml +1. **Share this checklist and failing tests** with the dev workflow (manual handoff) +2. **Review this checklist** with team in standup or planning +3. **Run failing tests** to confirm RED phase: `{test_command_all}` +4. **Begin implementation** using implementation checklist as guide +5. **Work one test at a time** (red → green for each) +6. **Share progress** in daily standup +7. **When all tests pass**, refactor code for quality +8. **When refactoring complete**, manually update story status to 'done' in sprint-status.yaml --- diff --git a/src/modules/bmm/workflows/testarch/atdd/checklist.md b/src/modules/bmm/workflows/testarch/atdd/checklist.md index 4430f665..ce94a14c 100644 --- a/src/modules/bmm/workflows/testarch/atdd/checklist.md +++ b/src/modules/bmm/workflows/testarch/atdd/checklist.md @@ -184,6 +184,7 @@ Before starting this workflow, verify: - [ ] Red-green-refactor workflow - [ ] Execution commands - [ ] Next steps for DEV team +- [ ] Output shared with DEV workflow (manual handoff; not auto-consumed) ### All Tests Verified to Fail (RED Phase) diff --git a/src/modules/bmm/workflows/testarch/atdd/instructions.md b/src/modules/bmm/workflows/testarch/atdd/instructions.md index 5fcb1328..aa748905 100644 --- a/src/modules/bmm/workflows/testarch/atdd/instructions.md +++ b/src/modules/bmm/workflows/testarch/atdd/instructions.md @@ -772,6 +772,7 @@ After completing this workflow, provide a summary: 5. Share progress in daily standup **Output File**: {output_file} +**Manual Handoff**: Share `{output_file}` and failing tests with the dev workflow (not auto-consumed). **Knowledge Base References Applied**: diff --git a/src/modules/bmm/workflows/testarch/automate/checklist.md b/src/modules/bmm/workflows/testarch/automate/checklist.md index 18290479..cc8c50a5 100644 --- a/src/modules/bmm/workflows/testarch/automate/checklist.md +++ b/src/modules/bmm/workflows/testarch/automate/checklist.md @@ -13,6 +13,7 @@ Before starting this workflow, verify: **Halt only if:** Framework scaffolding is completely missing (run `framework` workflow first) **Note:** BMad artifacts (story, tech-spec, PRD) are OPTIONAL - workflow can run without them +**Note:** `automate` generates tests; it does not run `*atdd` or `*test-review`. If ATDD outputs exist, use them as input and avoid duplicate coverage. --- @@ -421,6 +422,7 @@ Before starting this workflow, verify: **With atdd Workflow:** +- [ ] ATDD artifacts provided or located (manual handoff; `atdd` not auto-run) - [ ] Existing ATDD tests checked (if story had ATDD workflow run) - [ ] Expansion beyond ATDD planned (edge cases, negative paths) - [ ] No duplicate coverage with ATDD tests diff --git a/src/modules/bmm/workflows/testarch/ci/checklist.md b/src/modules/bmm/workflows/testarch/ci/checklist.md index 6853a24d..d68918fb 100644 --- a/src/modules/bmm/workflows/testarch/ci/checklist.md +++ b/src/modules/bmm/workflows/testarch/ci/checklist.md @@ -9,6 +9,8 @@ - [ ] Team agrees on CI platform - [ ] Access to CI platform settings (if updating) +Note: CI setup is typically a one-time task per repo and can be run any time after the test framework is configured. + ## Process Steps ### Step 1: Preflight Checks diff --git a/src/modules/bmm/workflows/testarch/ci/instructions.md b/src/modules/bmm/workflows/testarch/ci/instructions.md index acca1f80..a23d2c16 100644 --- a/src/modules/bmm/workflows/testarch/ci/instructions.md +++ b/src/modules/bmm/workflows/testarch/ci/instructions.md @@ -11,6 +11,8 @@ Scaffolds a production-ready CI/CD quality pipeline with test execution, burn-in loops for flaky test detection, parallel sharding, artifact collection, and notification configuration. This workflow creates platform-specific CI configuration optimized for fast feedback and reliable test execution. +Note: This is typically a one-time setup per repo; run it any time after the test framework exists, ideally before feature work starts. + --- ## Preflight Requirements diff --git a/src/modules/bmm/workflows/testarch/nfr-assess/checklist.md b/src/modules/bmm/workflows/testarch/nfr-assess/checklist.md index 44200b68..1e76f366 100644 --- a/src/modules/bmm/workflows/testarch/nfr-assess/checklist.md +++ b/src/modules/bmm/workflows/testarch/nfr-assess/checklist.md @@ -5,6 +5,8 @@ --- +Note: `nfr-assess` evaluates existing evidence; it does not run tests or CI workflows. + ## Prerequisites Validation - [ ] Implementation is deployed and accessible for evaluation diff --git a/src/modules/bmm/workflows/testarch/nfr-assess/nfr-report-template.md b/src/modules/bmm/workflows/testarch/nfr-assess/nfr-report-template.md index 2e9a133a..428a0a48 100644 --- a/src/modules/bmm/workflows/testarch/nfr-assess/nfr-report-template.md +++ b/src/modules/bmm/workflows/testarch/nfr-assess/nfr-report-template.md @@ -6,6 +6,8 @@ --- +Note: This assessment summarizes existing evidence; it does not run tests or CI workflows. + ## Executive Summary **Assessment:** {PASS_COUNT} PASS, {CONCERNS_COUNT} CONCERNS, {FAIL_COUNT} FAIL diff --git a/src/modules/bmm/workflows/testarch/test-design/checklist.md b/src/modules/bmm/workflows/testarch/test-design/checklist.md index cb89659f..24089fae 100644 --- a/src/modules/bmm/workflows/testarch/test-design/checklist.md +++ b/src/modules/bmm/workflows/testarch/test-design/checklist.md @@ -153,6 +153,7 @@ ### Workflow Dependencies - [ ] Can proceed to `atdd` workflow with P0 scenarios +- [ ] `atdd` is a separate workflow and must be run explicitly (not auto-run) - [ ] Can proceed to `automate` workflow with full coverage plan - [ ] Risk assessment informs `gate` workflow criteria - [ ] Integrates with `ci` workflow execution order @@ -176,7 +177,7 @@ 1. [ ] Review risk assessment with team 2. [ ] Prioritize mitigation for high-priority risks (score ≥6) 3. [ ] Allocate resources per estimates -4. [ ] Run `atdd` workflow to generate P0 tests +4. [ ] Run `atdd` workflow to generate P0 tests (separate workflow; not auto-run) 5. [ ] Set up test data factories and fixtures 6. [ ] Schedule team review of test design document diff --git a/src/modules/bmm/workflows/testarch/test-design/instructions.md b/src/modules/bmm/workflows/testarch/test-design/instructions.md index 1aa22a3d..14a2419e 100644 --- a/src/modules/bmm/workflows/testarch/test-design/instructions.md +++ b/src/modules/bmm/workflows/testarch/test-design/instructions.md @@ -764,7 +764,7 @@ After completing this workflow, provide a summary: 1. Review risk assessment with team 2. Prioritize mitigation for high-risk items (score ≥6) -3. Run `atdd` workflow to generate failing tests for P0 scenarios +3. Run `atdd` workflow to generate failing tests for P0 scenarios (separate workflow; not auto-run by `test-design`) 4. Allocate resources per effort estimates 5. Set up test data factories and fixtures ``` diff --git a/src/modules/bmm/workflows/testarch/test-design/test-design-template.md b/src/modules/bmm/workflows/testarch/test-design/test-design-template.md index 829e6e3a..cc3581ed 100644 --- a/src/modules/bmm/workflows/testarch/test-design/test-design-template.md +++ b/src/modules/bmm/workflows/testarch/test-design/test-design-template.md @@ -246,6 +246,15 @@ --- +--- + +## Follow-on Workflows (Manual) + +- Run `atdd` to generate failing P0 tests (separate workflow; not auto-run). +- Run `automate` for broader coverage once implementation exists. + +--- + ## Approval **Test Design Approved By:** diff --git a/src/modules/bmm/workflows/testarch/test-review/checklist.md b/src/modules/bmm/workflows/testarch/test-review/checklist.md index 7ff04863..f4fca8af 100644 --- a/src/modules/bmm/workflows/testarch/test-review/checklist.md +++ b/src/modules/bmm/workflows/testarch/test-review/checklist.md @@ -6,6 +6,8 @@ Use this checklist to validate that the test quality review workflow completed s ## Prerequisites +Note: `test-review` is optional and only audits existing tests; it does not generate tests. + ### Test File Discovery - [ ] Test file(s) identified for review (single/directory/suite scope) diff --git a/src/modules/bmm/workflows/testarch/test-review/test-review-template.md b/src/modules/bmm/workflows/testarch/test-review/test-review-template.md index 79bf9dea..54127a5a 100644 --- a/src/modules/bmm/workflows/testarch/test-review/test-review-template.md +++ b/src/modules/bmm/workflows/testarch/test-review/test-review-template.md @@ -7,6 +7,8 @@ --- +Note: This review audits existing tests; it does not generate tests. + ## Executive Summary **Overall Assessment**: {Excellent | Good | Acceptable | Needs Improvement | Critical Issues} diff --git a/src/modules/bmm/workflows/testarch/trace/checklist.md b/src/modules/bmm/workflows/testarch/trace/checklist.md index e9932367..ba4e63b8 100644 --- a/src/modules/bmm/workflows/testarch/trace/checklist.md +++ b/src/modules/bmm/workflows/testarch/trace/checklist.md @@ -16,6 +16,7 @@ This checklist covers **two sequential phases**: - [ ] Acceptance criteria are available (from story file OR inline) - [ ] Test suite exists (or gaps are acknowledged and documented) +- [ ] If tests are missing, recommend `*atdd` (trace does not run it automatically) - [ ] Test directory path is correct (`test_dir` variable) - [ ] Story file is accessible (if using BMad mode) - [ ] Knowledge base is loaded (test-priorities, traceability, risk-governance) diff --git a/src/modules/bmm/workflows/testarch/trace/instructions.md b/src/modules/bmm/workflows/testarch/trace/instructions.md index 77e930c1..bd575849 100644 --- a/src/modules/bmm/workflows/testarch/trace/instructions.md +++ b/src/modules/bmm/workflows/testarch/trace/instructions.md @@ -52,6 +52,8 @@ This workflow operates in two sequential phases to validate test coverage and de - If acceptance criteria are completely missing, halt and request them - If Phase 2 enabled but test execution results missing, warn and skip gate decision +Note: `*trace` never runs `*atdd` automatically; it only recommends running it when tests are missing. + --- ## PHASE 1: REQUIREMENTS TRACEABILITY diff --git a/src/modules/bmm/workflows/testarch/trace/trace-template.md b/src/modules/bmm/workflows/testarch/trace/trace-template.md index 2dc368c9..b2042eca 100644 --- a/src/modules/bmm/workflows/testarch/trace/trace-template.md +++ b/src/modules/bmm/workflows/testarch/trace/trace-template.md @@ -6,6 +6,8 @@ --- +Note: This workflow does not generate tests. If gaps exist, run `atdd` or `automate` to create coverage. + ## PHASE 1: REQUIREMENTS TRACEABILITY ### Coverage Summary