diff --git a/bmad/_cfg/files-manifest.csv b/bmad/_cfg/files-manifest.csv index 439c019c..f6adf593 100644 --- a/bmad/_cfg/files-manifest.csv +++ b/bmad/_cfg/files-manifest.csv @@ -35,6 +35,7 @@ type,name,module,path,hash "md","instructions","bmb","bmad/bmb/workflows/module-brief/instructions.md","e2275373850ea0745f396ad0c3aa192f06081b52d98777650f6b645333b62926" "md","instructions","bmb","bmad/bmb/workflows/redoc/instructions.md","21dd93b64455f8dd475b508ae9f1076d7e179e99fb6f197476071706b78e3592" "md","module-structure","bmb","bmad/bmb/workflows/create-module/module-structure.md","3bdf1d55eec2fccc2c9f44a08f4e0dc489ce47396ff39fa59a82836a911faa54" +"md","tea-README","bmm","bmad/bmm/docs/tea-README.md","2ae906adc1edde5ba3af2a20d78d9cef640897347ec46453233d611115c3e1ac" "md","README","bmb","bmad/bmb/README.md","aa2beac1fb84267cbaa6d7eb541da824c34177a17cd227f11b189ab3a1e06d33" "md","README","bmb","bmad/bmb/workflows/convert-legacy/README.md","2c11bcf8d974e4f0e0e03f948df42097592751a3aeb9c443fa6cecf05819d49b" "md","README","bmb","bmad/bmb/workflows/create-agent/README.md","f4da5c16fb4847252b09b82d70f027ae08e78b75bb101601f2ca3d2c2c884736" diff --git a/bmad/bmm/docs/README.md b/bmad/bmm/docs/README.md index 0b483833..6308475a 100644 --- a/bmad/bmm/docs/README.md +++ b/bmad/bmm/docs/README.md @@ -156,7 +156,7 @@ For detailed technical documentation on specific complex workflows: Quality assurance guidance: -- **[Test Architect Guide](../testarch/README.md)** - Comprehensive testing strategy +- **[Test Architect Guide](./tea-README.md)** - Comprehensive testing strategy - Test design workflows - Quality gates - Risk assessment diff --git a/bmad/bmm/docs/brownfield-guide.md b/bmad/bmm/docs/brownfield-guide.md index e185db2d..ad564c8d 100644 --- a/bmad/bmm/docs/brownfield-guide.md +++ b/bmad/bmm/docs/brownfield-guide.md @@ -751,7 +751,7 @@ flowchart TD **Documentation:** - [BMM Workflows Guide](../workflows/README.md) -- [Test Architect Guide](../testarch/README.md) +- [Test Architect Guide](./tea-README.md) - [BMM Module README](../README.md) --- diff --git a/bmad/bmm/docs/tea-README.md b/bmad/bmm/docs/tea-README.md new file mode 100644 index 00000000..efda1375 --- /dev/null +++ b/bmad/bmm/docs/tea-README.md @@ -0,0 +1,311 @@ +--- +last-redoc-date: 2025-10-14 +--- + +# Test Architect (TEA) Agent Guide + +## Overview + +- **Persona:** Murat, Master Test Architect and Quality Advisor focused on risk-based testing, fixture architecture, ATDD, and CI/CD governance. +- **Mission:** Deliver actionable quality strategies, automation coverage, and gate decisions that scale with project level and compliance demands. +- **Use When:** Project level ≥2, integration risk is non-trivial, brownfield regression risk exists, or compliance/NFR evidence is required. + +## TEA Workflow Lifecycle + +TEA integrates across the entire BMad development lifecycle, providing quality assurance at every phase: + +``` +┌──────────────────────────────────────────────────────────┐ +│ BMM Phase 2: PLANNING │ +│ │ +│ PM: *prd │ +│ ↓ │ +│ TEA: *framework ──→ *ci ──→ *test-design │ +│ └─────────┬─────────────┘ │ +│ │ (Setup once per project) │ +└─────────────────┼──────────────────────────────────────────┘ + ↓ +┌──────────────────────────────────────────────────────────┐ +│ BMM Phase 4: IMPLEMENTATION │ +│ (Per Story Cycle) │ +│ │ +│ ┌─→ SM: *create-story │ +│ │ ↓ │ +│ │ TEA: *atdd (optional, before dev) │ +│ │ ↓ │ +│ │ DEV: implements story │ +│ │ ↓ │ +│ │ TEA: *automate ──→ *test-review (optional) │ +│ │ ↓ │ +│ │ TEA: *trace (refresh coverage) │ +│ │ ↓ │ +│ └───[next story] │ +└─────────────────┼──────────────────────────────────────────┘ + ↓ +┌──────────────────────────────────────────────────────────┐ +│ EPIC/RELEASE GATE │ +│ │ +│ TEA: *nfr-assess (if not done earlier) │ +│ ↓ │ +│ TEA: *test-review (final audit, optional) │ +│ ↓ │ +│ TEA: *trace (Phase 2: Gate) ──→ PASS | CONCERNS | FAIL | WAIVED │ +│ │ +└──────────────────────────────────────────────────────────┘ +``` + +### TEA Integration with BMad v6 Workflow + +TEA operates **across all four BMad phases**, unlike other agents that are phase-specific: + +
+Cross-Phase Integration & Workflow Complexity + +### Phase-Specific Agents (Standard Pattern) + +- **Phase 1 (Analysis)**: Analyst agent +- **Phase 2 (Planning)**: PM agent +- **Phase 3 (Solutioning)**: Architect agent +- **Phase 4 (Implementation)**: SM, DEV agents + +### TEA: Cross-Phase Quality Agent (Unique Pattern) + +TEA is **the only agent that spans all phases**: + +``` +Phase 1 (Analysis) → [TEA not typically used] + ↓ +Phase 2 (Planning) → TEA: *framework, *ci, *test-design (setup) + ↓ +Phase 3 (Solutioning) → [TEA validates architecture testability] + ↓ +Phase 4 (Implementation) → TEA: *atdd, *automate, *test-review, *trace (per story) + ↓ +Epic/Release Gate → TEA: *nfr-assess, *trace Phase 2 (release decision) +``` + +### Why TEA Needs 8 Workflows + +**Standard agents**: 1-3 workflows per phase +**TEA**: 8 workflows across 3+ phases + +| Phase | TEA Workflows | Frequency | Purpose | +| ----------- | -------------------------------------- | ---------------- | -------------------------------- | +| **Phase 2** | *framework, *ci, \*test-design | Once per project | Establish quality infrastructure | +| **Phase 4** | *atdd, *automate, *test-review, *trace | Per story/sprint | Continuous quality validation | +| **Release** | *nfr-assess, *trace (Phase 2: gate) | Per epic/release | Go/no-go decision | + +**Note**: `*trace` is a two-phase workflow: Phase 1 (traceability) + Phase 2 (gate decision). This reduces cognitive load while maintaining natural workflow. + +This complexity **requires specialized documentation** (this guide), **extensive knowledge base** (19+ fragments), and **unique architecture** (`testarch/` directory). + +
+ +## Prerequisites and Setup + +1. Run the core planning workflows first: + - Analyst `*product-brief` + - Product Manager `*prd` + - Architect `*create-architecture` +2. Confirm `bmad/bmm/config.yaml` defines `project_name`, `output_folder`, `dev_story_location`, and language settings. +3. Ensure a test test framework setup exists; if not, use `*framework` command to create a test framework setup, prior to development. +4. Skim supporting references (knowledge under `testarch/`, command workflows under `workflows/testarch/`). + - `tea-index.csv` + `knowledge/*.md` + +## High-Level Cheat Sheets + +### Greenfield Feature Launch (Level 2) + +| Phase | Test Architect | Dev / Team | Outputs | +| ------------------ | ------------------------------------------------------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------------------- | +| Setup | - | Analyst `*product-brief`, PM `*prd`, Architect `*create-architecture` | `{output_folder}/product-brief*.md`, `PRD.md`, `epics.md`, `architecture.md` | +| Pre-Implementation | Run `*framework` (if harness missing), `*ci`, and `*test-design` | Review risk/design/CI guidance, align backlog | Test scaffold, CI pipeline, risk and coverage strategy | +| Story Prep | - | Scrum Master `*create-story`, `*story-context` | Story markdown + context XML | +| Implementation | (Optional) Trigger `*atdd` before dev to supply failing tests + checklist | Implement story guided by ATDD checklist | Failing acceptance tests + implementation checklist | +| Post-Dev | Execute `*automate`, (Optional) `*test-review`, re-run `*trace` | Address recommendations, update code/tests | Regression specs, quality report, refreshed coverage matrix | +| Release | (Optional) `*test-review` for final audit, Run `*trace` (Phase 2) | Confirm Definition of Done, share release notes | Quality audit, Gate YAML + release summary (owners, waivers) | + +
+Execution Notes + +- Run `*framework` only once per repo or when modern harness support is missing. +- `*framework` followed by `*ci` establishes install + pipeline; `*test-design` then handles risk scoring, mitigations, and scenario planning in one pass. +- Use `*atdd` before coding when the team can adopt ATDD; share its checklist with the dev agent. +- Post-implementation, keep `*trace` current, expand coverage with `*automate`, optionally review test quality with `*test-review`. For release gate, run `*trace` with Phase 2 enabled to get deployment decision. +- Use `*test-review` after `*atdd` to validate generated tests, after `*automate` to ensure regression quality, or before gate for final audit. + +
+ +
+Worked Example – “Nova CRM” Greenfield Feature + +1. **Planning:** Analyst runs `*product-brief`; PM executes `*prd` to produce PRD and epics; Architect completes `*create-architecture` for the new module. +2. **Setup:** TEA checks harness via `*framework`, configures `*ci`, and runs `*test-design` to capture risk/coverage plans. +3. **Story Prep:** Scrum Master generates the story via `*create-story`; PO validates using `*solutioning-gate-check`. +4. **Implementation:** TEA optionally runs `*atdd`; Dev implements with guidance from failing tests and the plan. +5. **Post-Dev and Release:** TEA runs `*automate`, optionally `*test-review` to audit test quality, re-runs `*trace` with Phase 2 enabled to generate both traceability and gate decision. + +
+ +### Brownfield Feature Enhancement (Level 3–4) + +| Phase | Test Architect | Dev / Team | Outputs | +| ----------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------- | +| Refresh Context | - | Analyst/PM/Architect rerun planning workflows | Updated planning artifacts in `{output_folder}` | +| Baseline Coverage | Run `*trace` to inventory existing tests | Review matrix, flag hotspots | Coverage matrix + initial gate snippet | +| Risk Targeting | Run `*test-design` | Align remediation/backlog priorities | Brownfield risk memo + scenario matrix | +| Story Prep | - | Scrum Master `*create-story` | Updated story markdown | +| Implementation | (Optional) Run `*atdd` before dev | Implement story, referencing checklist/tests | Failing acceptance tests + implementation checklist | +| Post-Dev | Apply `*automate`, (Optional) `*test-review`, re-run `*trace`, `*nfr-assess` if needed | Resolve gaps, update docs/tests | Regression specs, quality report, refreshed coverage matrix, NFR report | +| Release | (Optional) `*test-review` for final audit, Run `*trace` (Phase 2) | Product Owner `*solutioning-gate-check`, share release notes | Quality audit, Gate YAML + release summary | + +
+Execution Notes + +- Lead with `*trace` so remediation plans target true coverage gaps. Ensure `*framework` and `*ci` are in place early in the engagement; if the brownfield lacks them, run those setup steps immediately after refreshing context. +- `*test-design` should highlight regression hotspots, mitigations, and P0 scenarios. +- Use `*atdd` when stories benefit from ATDD; otherwise proceed to implementation and rely on post-dev automation. +- After development, expand coverage with `*automate`, optionally review test quality with `*test-review`, re-run `*trace` (Phase 2 for gate decision). Run `*nfr-assess` now if non-functional risks weren't addressed earlier. +- Use `*test-review` to validate existing brownfield tests or audit new tests before gate. +- Product Owner `*solutioning-gate-check` confirms the team has artifacts before handoff or release. + +
+ +
+Worked Example – “Atlas Payments” Brownfield Story + +1. **Context Refresh:** Analyst reruns `*product-brief`; PM executes `*prd` to update PRD, analysis, and `epics.md`; Architect triggers `*create-architecture` capturing legacy payment flows. +2. **Baseline Coverage:** TEA executes `*trace` to record current coverage in `docs/qa/assessments/atlas-payment-trace.md`. +3. **Risk and Design:** `*test-design` flags settlement edge cases, plans mitigations, and allocates new API/E2E scenarios with P0 priorities. +4. **Story Prep:** Scrum Master generates `stories/story-1.1.md` via `*create-story`, automatically pulling updated context. +5. **ATDD First:** TEA runs `*atdd`, producing failing Playwright specs under `tests/e2e/payments/` plus an implementation checklist. +6. **Implementation:** Dev pairs with the checklist/tests to deliver the story. +7. **Post-Implementation:** TEA applies `*automate`, optionally `*test-review` to audit test quality, re-runs `*trace` with Phase 2 enabled, performs `*nfr-assess` to validate SLAs. The `*trace` Phase 2 output marks PASS with follow-ups. + +
+ +### Enterprise / Compliance Program (Level 4) + +| Phase | Test Architect | Dev / Team | Outputs | +| ------------------- | ----------------------------------------------------------------- | ---------------------------------------------- | ---------------------------------------------------------- | +| Strategic Planning | - | Analyst/PM/Architect standard workflows | Enterprise-grade PRD, epics, architecture | +| Quality Planning | Run `*framework`, `*test-design`, `*nfr-assess` | Review guidance, align compliance requirements | Harness scaffold, risk + coverage plan, NFR documentation | +| Pipeline Enablement | Configure `*ci` | Coordinate secrets, pipeline approvals | `.github/workflows/test.yml`, helper scripts | +| Execution | Enforce `*atdd`, `*automate`, `*test-review`, `*trace` per story | Implement stories, resolve TEA findings | Tests, fixtures, quality reports, coverage matrices | +| Release | (Optional) `*test-review` for final audit, Run `*trace` (Phase 2) | Capture sign-offs, archive artifacts | Quality audit, updated assessments, gate YAML, audit trail | + +
+Execution Notes + +- Use `*atdd` for every story when feasible so acceptance tests lead implementation in regulated environments. +- `*ci` scaffolds selective testing scripts, burn-in jobs, caching, and notifications for long-running suites. +- Enforce `*test-review` per story or sprint to maintain quality standards and ensure compliance with testing best practices. +- Prior to release, rerun coverage (`*trace`, `*automate`), perform final quality audit with `*test-review`, and formalize the decision with `*trace` Phase 2 (gate decision); store everything for audits. Call `*nfr-assess` here if compliance/performance requirements weren't captured during planning. + +
+ +
+Worked Example – “Helios Ledger” Enterprise Release + +1. **Strategic Planning:** Analyst/PM/Architect complete PRD, epics, and architecture using the standard workflows. +2. **Quality Planning:** TEA runs `*framework`, `*test-design`, and `*nfr-assess` to establish mitigations, coverage, and NFR targets. +3. **Pipeline Setup:** TEA configures CI via `*ci` with selective execution scripts. +4. **Execution:** For each story, TEA enforces `*atdd`, `*automate`, `*test-review`, and `*trace`; Dev teams iterate on the findings. +5. **Release:** TEA re-checks coverage, performs final quality audit with `*test-review`, and logs the final gate decision via `*trace` Phase 2, archiving artifacts for compliance. + +
+ +## Command Catalog + +
+Optional Playwright MCP Enhancements + +**Two Playwright MCP servers** (actively maintained, continuously updated): + +- `playwright` - Browser automation (`npx @playwright/mcp@latest`) +- `playwright-test` - Test runner with failure analysis (`npx playwright run-test-mcp-server`) + +**How MCP Enhances TEA Workflows**: + +MCP provides additional capabilities on top of TEA's default AI-based approach: + +1. `*test-design`: + - Default: Analysis + documentation + - **+ MCP**: Interactive UI discovery with `browser_navigate`, `browser_click`, `browser_snapshot`, behavior observation + + Benefit:Discover actual functionality, edge cases, undocumented features + +2. `*atdd`, `*automate`: + - Default: Infers selectors and interactions from requirements and knowledge fragments + - **+ MCP**: Generates tests **then** verifies with `generator_setup_page`, `browser_*` tools, validates against live app + + Benefit: Accurate selectors from real DOM, verified behavior, refined test code + +3. `*automate`: + - Default: Pattern-based fixes from error messages + knowledge fragments + - **+ MCP**: Pattern fixes **enhanced with** `browser_snapshot`, `browser_console_messages`, `browser_network_requests`, `browser_generate_locator` + + Benefit: Visual failure context, live DOM inspection, root cause discovery + +**Config example**: + +```json +{ + "mcpServers": { + "playwright": { + "command": "npx", + "args": ["@playwright/mcp@latest"] + }, + "playwright-test": { + "command": "npx", + "args": ["playwright", "run-test-mcp-server"] + } + } +} +``` + +**To disable**: Set `tea_use_mcp_enhancements: false` in `bmad/bmm/config.yaml` OR remove MCPs from IDE config. + +
+ +

+ +| Command | Workflow README | Primary Outputs | Notes | With Playwright MCP Enhancements | +| -------------- | ------------------------------------------------- | --------------------------------------------------------------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | +| `*framework` | [📖](../workflows/testarch/framework/README.md) | Playwright/Cypress scaffold, `.env.example`, `.nvmrc`, sample specs | Use when no production-ready harness exists | - | +| `*ci` | [📖](../workflows/testarch/ci/README.md) | CI workflow, selective test scripts, secrets checklist | Platform-aware (GitHub Actions default) | - | +| `*test-design` | [📖](../workflows/testarch/test-design/README.md) | Combined risk assessment, mitigation plan, and coverage strategy | Risk scoring + optional exploratory mode | **+ Exploratory**: Interactive UI discovery with browser automation (uncover actual functionality) | +| `*atdd` | [📖](../workflows/testarch/atdd/README.md) | Failing acceptance tests + implementation checklist | TDD red phase + optional recording mode | **+ Recording**: AI generation verified with live browser (accurate selectors from real DOM) | +| `*automate` | [📖](../workflows/testarch/automate/README.md) | Prioritized specs, fixtures, README/script updates, DoD summary | Optional healing/recording, avoid duplicate coverage | **+ Healing**: Pattern fixes enhanced with visual debugging + **+ Recording**: AI verified with live browser | +| `*test-review` | [📖](../workflows/testarch/test-review/README.md) | Test quality review report with 0-100 score, violations, fixes | Reviews tests against knowledge base patterns | - | +| `*nfr-assess` | [📖](../workflows/testarch/nfr-assess/README.md) | NFR assessment report with actions | Focus on security/performance/reliability | - | +| `*trace` | [📖](../workflows/testarch/trace/README.md) | Phase 1: Coverage matrix, recommendations. Phase 2: Gate decision (PASS/CONCERNS/FAIL/WAIVED) | Two-phase workflow: traceability + gate decision | - | + +**📖** = Click to view detailed workflow documentation + +## Why TEA is Architecturally Different + +TEA is the only BMM agent with its own top-level module directory (`bmm/testarch/`). This intentional design pattern reflects TEA's unique requirements: + +
+Unique Architecture Pattern & Rationale + +### Directory Structure + +``` +src/modules/bmm/ +├── agents/ +│ └── tea.agent.yaml # Agent definition (standard location) +├── workflows/ +│ └── testarch/ # TEA workflows (standard location) +└── testarch/ # Knowledge base (UNIQUE!) + ├── knowledge/ # 21 production-ready test pattern fragments + ├── tea-index.csv # Centralized knowledge lookup (21 fragments indexed) + └── README.md # This guide +``` + +### Why TEA Gets Special Treatment + +TEA uniquely requires **extensive domain knowledge** (21 fragments, 12,821 lines: test patterns, CI/CD, fixtures, quality practices, healing strategies), a **centralized reference system** (`tea-index.csv` for on-demand fragment loading), **cross-cutting concerns** (domain-specific patterns vs project-specific artifacts like PRDs/stories), and **optional MCP integration** (healing, exploratory, verification modes). Other BMM agents don't require this architecture. + +
diff --git a/bmad/bmm/workflows/testarch/atdd/atdd-checklist-template.md b/bmad/bmm/workflows/testarch/atdd/atdd-checklist-template.md index 027eb18e..a64c4969 100644 --- a/bmad/bmm/workflows/testarch/atdd/atdd-checklist-template.md +++ b/bmad/bmm/workflows/testarch/atdd/atdd-checklist-template.md @@ -355,8 +355,8 @@ See `tea-index.csv` for complete knowledge fragment mapping. - Ask in team standup - Tag @{tea_agent_username} in Slack/Discord -- Refer to `testarch/README.md` for workflow documentation -- Consult `testarch/knowledge/` for testing best practices +- Refer to `./bmm/docs/tea-README.md` for workflow documentation +- Consult `./bmm/testarch/knowledge` for testing best practices --- diff --git a/package-lock.json b/package-lock.json index adf6696e..4de233c1 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "bmad-method", - "version": "6.0.0-alpha.3", + "version": "6.0.0-alpha.5", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "bmad-method", - "version": "6.0.0-alpha.3", + "version": "6.0.0-alpha.5", "license": "MIT", "dependencies": { "@kayvan/markdown-tree-parser": "^1.6.1", diff --git a/src/modules/bmm/docs/README.md b/src/modules/bmm/docs/README.md index 0b483833..5a556dac 100644 --- a/src/modules/bmm/docs/README.md +++ b/src/modules/bmm/docs/README.md @@ -127,7 +127,7 @@ Comprehensive documentation for all BMM workflows organized by phase: - Complete story lifecycle - One-story-at-a-time discipline -- **[Testing & QA Workflows](./workflows-testing.md)** - Comprehensive quality assurance (1,420 lines) +- **[Testing & QA Workflows](./tea-README.md)** - Comprehensive quality assurance (1,420 lines) - Test strategy, automation, quality gates - TEA agent and test healing - BMad-integrated vs standalone modes diff --git a/src/modules/bmm/docs/brownfield-guide.md b/src/modules/bmm/docs/brownfield-guide.md index e185db2d..06c1a69e 100644 --- a/src/modules/bmm/docs/brownfield-guide.md +++ b/src/modules/bmm/docs/brownfield-guide.md @@ -751,7 +751,7 @@ flowchart TD **Documentation:** - [BMM Workflows Guide](../workflows/README.md) -- [Test Architect Guide](../testarch/README.md) +- [Test Architect Guide](../../../../bmad/bmm/docs/tea-README.md) - [BMM Module README](../README.md) --- diff --git a/src/modules/bmm/workflows/testarch/atdd/atdd-checklist-template.md b/src/modules/bmm/workflows/testarch/atdd/atdd-checklist-template.md index 027eb18e..a64c4969 100644 --- a/src/modules/bmm/workflows/testarch/atdd/atdd-checklist-template.md +++ b/src/modules/bmm/workflows/testarch/atdd/atdd-checklist-template.md @@ -355,8 +355,8 @@ See `tea-index.csv` for complete knowledge fragment mapping. - Ask in team standup - Tag @{tea_agent_username} in Slack/Discord -- Refer to `testarch/README.md` for workflow documentation -- Consult `testarch/knowledge/` for testing best practices +- Refer to `./bmm/docs/tea-README.md` for workflow documentation +- Consult `./bmm/testarch/knowledge` for testing best practices --- diff --git a/tools/schema/agent.js b/tools/schema/agent.js index a7c4671a..704600ad 100644 --- a/tools/schema/agent.js +++ b/tools/schema/agent.js @@ -176,6 +176,8 @@ function buildMenuItemSchema() { tmpl: createNonEmptyString('agent.menu[].tmpl').optional(), data: createNonEmptyString('agent.menu[].data').optional(), 'run-workflow': createNonEmptyString('agent.menu[].run-workflow').optional(), + checklist: createNonEmptyString('agent.menu[].checklist').optional(), + document: createNonEmptyString('agent.menu[].document').optional(), }) .strict() .superRefine((value, ctx) => {