Compare commits
33 Commits
1d49e045db
...
36dfb0a612
| Author | SHA1 | Date |
|---|---|---|
|
|
36dfb0a612 | |
|
|
28e6dded4d | |
|
|
966ca5db0b | |
|
|
e0318d9da8 | |
|
|
4a983d64a7 | |
|
|
f25fcc686c | |
|
|
411cded4d0 | |
|
|
a50d82df1c | |
|
|
d022e569bd | |
|
|
7990ad528c | |
|
|
5881790068 | |
|
|
d83a88da66 | |
|
|
7b68d1a326 | |
|
|
7cd4926adb | |
|
|
0fa53ad144 | |
|
|
afee68ca99 | |
|
|
b952d28fb3 | |
|
|
577c1aa218 | |
|
|
abba7ee987 | |
|
|
d34efa2695 | |
|
|
87b1292e3f | |
|
|
43f7eee29a | |
|
|
96f21be73e | |
|
|
66e7d3a36d | |
|
|
2b7f7ff421 | |
|
|
3360666c2a | |
|
|
274dea16fa | |
|
|
dcd581c84a | |
|
|
6d84a60a78 | |
|
|
59e1b7067c | |
|
|
1d8df63ac5 | |
|
|
993d02b8b3 | |
|
|
5cb5606ba3 |
|
|
@ -10,6 +10,7 @@ permissions:
|
|||
|
||||
jobs:
|
||||
bundle-and-publish:
|
||||
if: ${{ false }} # Temporarily disabled while web bundles are paused.
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout BMAD-METHOD
|
||||
|
|
|
|||
|
|
@ -69,6 +69,27 @@ jobs:
|
|||
- name: markdownlint
|
||||
run: npm run lint:md
|
||||
|
||||
docs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
cache: "npm"
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Validate documentation links
|
||||
run: npm run docs:validate-links
|
||||
|
||||
- name: Build documentation
|
||||
run: npm run docs:build
|
||||
|
||||
validate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
|
|
|
|||
|
|
@ -44,22 +44,15 @@ CLAUDE.local.md
|
|||
.claude/settings.local.json
|
||||
|
||||
# Project-specific
|
||||
_bmad-core
|
||||
_bmad-creator-tools
|
||||
flattened-codebase.xml
|
||||
*.stats.md
|
||||
.internal-docs/
|
||||
#UAT template testing output files
|
||||
tools/template-test-generator/test-scenarios/
|
||||
|
||||
# Bundler temporary files and generated bundles
|
||||
.bundler-temp/
|
||||
web-bundles/
|
||||
|
||||
# Generated web bundles (built by CI, not committed)
|
||||
src/modules/bmm/sub-modules/
|
||||
src/modules/bmb/sub-modules/
|
||||
src/modules/cis/sub-modules/
|
||||
src/modules/bmgd/sub-modules/
|
||||
shared-modules
|
||||
z*/
|
||||
|
||||
|
|
@ -68,6 +61,7 @@ _bmad-output
|
|||
.claude
|
||||
.codex
|
||||
.github/chatmodes
|
||||
.github/agents
|
||||
.agent
|
||||
.agentvibes/
|
||||
.kiro/
|
||||
|
|
|
|||
|
|
@ -5,3 +5,16 @@ npx --no-install lint-staged
|
|||
|
||||
# Validate everything
|
||||
npm test
|
||||
|
||||
# Validate docs links only when docs change
|
||||
if command -v rg >/dev/null 2>&1; then
|
||||
if git diff --cached --name-only | rg -q '^docs/'; then
|
||||
npm run docs:validate-links
|
||||
npm run docs:build
|
||||
fi
|
||||
else
|
||||
if git diff --cached --name-only | grep -Eq '^docs/'; then
|
||||
npm run docs:validate-links
|
||||
npm run docs:build
|
||||
fi
|
||||
fi
|
||||
|
|
|
|||
|
|
@ -11,7 +11,6 @@ ignores:
|
|||
- .claude/**
|
||||
- .roo/**
|
||||
- .codex/**
|
||||
- .agentvibes/**
|
||||
- .kiro/**
|
||||
- sample-project/**
|
||||
- test-project-install/**
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# BMad Method
|
||||

|
||||
|
||||
[](https://www.npmjs.com/package/bmad-method)
|
||||
[](LICENSE)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,85 @@
|
|||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
We release security patches for the following versions:
|
||||
|
||||
| Version | Supported |
|
||||
| ------- | ------------------ |
|
||||
| Latest | :white_check_mark: |
|
||||
| < Latest | :x: |
|
||||
|
||||
We recommend always using the latest version of BMad Method to ensure you have the most recent security updates.
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
We take security vulnerabilities seriously. If you discover a security issue, please report it responsibly.
|
||||
|
||||
### How to Report
|
||||
|
||||
**Do NOT report security vulnerabilities through public GitHub issues.**
|
||||
|
||||
Instead, please report them via one of these methods:
|
||||
|
||||
1. **GitHub Security Advisories** (Preferred): Use [GitHub's private vulnerability reporting](https://github.com/bmad-code-org/BMAD-METHOD/security/advisories/new) to submit a confidential report.
|
||||
|
||||
2. **Discord**: Contact a maintainer directly via DM on our [Discord server](https://discord.gg/gk8jAdXWmj).
|
||||
|
||||
### What to Include
|
||||
|
||||
Please include as much of the following information as possible:
|
||||
|
||||
- Type of vulnerability (e.g., prompt injection, path traversal, etc.)
|
||||
- Full paths of source file(s) related to the vulnerability
|
||||
- Step-by-step instructions to reproduce the issue
|
||||
- Proof-of-concept or exploit code (if available)
|
||||
- Impact assessment of the vulnerability
|
||||
|
||||
### Response Timeline
|
||||
|
||||
- **Initial Response**: Within 48 hours of receiving your report
|
||||
- **Status Update**: Within 7 days with our assessment
|
||||
- **Resolution Target**: Critical issues within 30 days; other issues within 90 days
|
||||
|
||||
### What to Expect
|
||||
|
||||
1. We will acknowledge receipt of your report
|
||||
2. We will investigate and validate the vulnerability
|
||||
3. We will work on a fix and coordinate disclosure timing with you
|
||||
4. We will credit you in the security advisory (unless you prefer to remain anonymous)
|
||||
|
||||
## Security Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- Vulnerabilities in BMad Method core framework code
|
||||
- Security issues in agent definitions or workflows that could lead to unintended behavior
|
||||
- Path traversal or file system access issues
|
||||
- Prompt injection vulnerabilities that bypass intended agent behavior
|
||||
- Supply chain vulnerabilities in dependencies
|
||||
|
||||
### Out of Scope
|
||||
|
||||
- Security issues in user-created custom agents or modules
|
||||
- Vulnerabilities in third-party AI providers (Claude, GPT, etc.)
|
||||
- Issues that require physical access to a user's machine
|
||||
- Social engineering attacks
|
||||
- Denial of service attacks that don't exploit a specific vulnerability
|
||||
|
||||
## Security Best Practices for Users
|
||||
|
||||
When using BMad Method:
|
||||
|
||||
1. **Review Agent Outputs**: Always review AI-generated code before executing it
|
||||
2. **Limit File Access**: Configure your AI IDE to limit file system access where possible
|
||||
3. **Keep Updated**: Regularly update to the latest version
|
||||
4. **Validate Dependencies**: Review any dependencies added by generated code
|
||||
5. **Environment Isolation**: Consider running AI-assisted development in isolated environments
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
We appreciate the security research community's efforts in helping keep BMad Method secure. Contributors who report valid security issues will be acknowledged in our security advisories.
|
||||
|
||||
---
|
||||
|
||||
Thank you for helping keep BMad Method and our community safe.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 23 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 366 KiB |
|
|
@ -2,416 +2,304 @@
|
|||
title: "Documentation Style Guide"
|
||||
---
|
||||
|
||||
Internal guidelines for maintaining consistent, high-quality documentation across the BMad Method project. This document is not included in the Starlight sidebar — it's for contributors and maintainers, not end users.
|
||||
This project adheres to the [Google Developer Documentation Style Guide](https://developers.google.com/style) and uses [Diataxis](https://diataxis.fr/) to structure content. Only project-specific conventions follow.
|
||||
|
||||
## Quick Principles
|
||||
## Project-Specific Rules
|
||||
|
||||
1. **Clarity over brevity** — Be concise, but never at the cost of understanding
|
||||
2. **Consistent structure** — Follow established patterns so readers know what to expect
|
||||
3. **Strategic visuals** — Use admonitions, tables, and diagrams purposefully
|
||||
4. **Scannable content** — Headers, lists, and callouts help readers find what they need
|
||||
| Rule | Specification |
|
||||
|------|---------------|
|
||||
| No horizontal rules (`---`) | Fragments reading flow |
|
||||
| No `####` headers | Use bold text or admonitions instead |
|
||||
| No "Related" or "Next:" sections | Sidebar handles navigation |
|
||||
| No deeply nested lists | Break into sections instead |
|
||||
| No code blocks for non-code | Use admonitions for dialogue examples |
|
||||
| No bold paragraphs for callouts | Use admonitions instead |
|
||||
| 1-2 admonitions per section max | Tutorials allow 3-4 per major section |
|
||||
| Table cells / list items | 1-2 sentences max |
|
||||
| Header budget | 8-12 `##` per doc; 2-3 `###` per section |
|
||||
|
||||
## Validation Steps
|
||||
## Admonitions (Starlight Syntax)
|
||||
|
||||
Before submitting documentation changes, run these checks from the repo root:
|
||||
```md
|
||||
:::tip[Title]
|
||||
Shortcuts, best practices
|
||||
:::
|
||||
|
||||
1. **Fix link format** — Convert relative links (`./`, `../`) to site-relative paths (`/path/`)
|
||||
```bash
|
||||
npm run docs:fix-links # Preview changes
|
||||
npm run docs:fix-links -- --write # Apply changes
|
||||
```
|
||||
:::note[Title]
|
||||
Context, definitions, examples, prerequisites
|
||||
:::
|
||||
|
||||
2. **Validate links** — Check all links point to existing files
|
||||
```bash
|
||||
npm run docs:validate-links # Preview issues
|
||||
npm run docs:validate-links -- --write # Auto-fix where possible
|
||||
```
|
||||
:::caution[Title]
|
||||
Caveats, potential issues
|
||||
:::
|
||||
|
||||
3. **Build the site** — Verify no build errors
|
||||
```bash
|
||||
npm run docs:build
|
||||
```
|
||||
:::danger[Title]
|
||||
Critical warnings only — data loss, security issues
|
||||
:::
|
||||
```
|
||||
|
||||
### Standard Uses
|
||||
|
||||
| Admonition | Use For |
|
||||
|------------|---------|
|
||||
| `:::note[Prerequisites]` | Dependencies before starting |
|
||||
| `:::tip[Quick Path]` | TL;DR summary at document top |
|
||||
| `:::caution[Important]` | Critical caveats |
|
||||
| `:::note[Example]` | Command/response examples |
|
||||
|
||||
## Standard Table Formats
|
||||
|
||||
**Phases:**
|
||||
|
||||
```md
|
||||
| Phase | Name | What Happens |
|
||||
|-------|------|--------------|
|
||||
| 1 | Analysis | Brainstorm, research *(optional)* |
|
||||
| 2 | Planning | Requirements — PRD or tech-spec *(required)* |
|
||||
```
|
||||
|
||||
**Commands:**
|
||||
|
||||
```md
|
||||
| Command | Agent | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `*workflow-init` | Analyst | Initialize a new project |
|
||||
| `*prd` | PM | Create Product Requirements Document |
|
||||
```
|
||||
|
||||
## Folder Structure Blocks
|
||||
|
||||
Show in "What You've Accomplished" sections:
|
||||
|
||||
````md
|
||||
```
|
||||
your-project/
|
||||
├── _bmad/ # BMad configuration
|
||||
├── _bmad-output/
|
||||
│ ├── PRD.md # Your requirements document
|
||||
│ └── bmm-workflow-status.yaml # Progress tracking
|
||||
└── ...
|
||||
```
|
||||
````
|
||||
|
||||
## Tutorial Structure
|
||||
|
||||
Every tutorial should follow this structure:
|
||||
|
||||
```
|
||||
1. Title + Hook (1-2 sentences describing the outcome)
|
||||
2. Version/Module Notice (info or warning admonition as appropriate)
|
||||
```text
|
||||
1. Title + Hook (1-2 sentences describing outcome)
|
||||
2. Version/Module Notice (info or warning admonition) (optional)
|
||||
3. What You'll Learn (bullet list of outcomes)
|
||||
4. Prerequisites (info admonition)
|
||||
5. Quick Path (tip admonition - TL;DR summary)
|
||||
6. Understanding [Topic] (context before steps - tables for phases/agents)
|
||||
7. Installation (if applicable)
|
||||
7. Installation (optional)
|
||||
8. Step 1: [First Major Task]
|
||||
9. Step 2: [Second Major Task]
|
||||
10. Step 3: [Third Major Task]
|
||||
11. What You've Accomplished (summary + folder structure if applicable)
|
||||
11. What You've Accomplished (summary + folder structure)
|
||||
12. Quick Reference (commands table)
|
||||
13. Common Questions (FAQ format)
|
||||
14. Getting Help (community links)
|
||||
15. Key Takeaways (tip admonition - memorable points)
|
||||
15. Key Takeaways (tip admonition)
|
||||
```
|
||||
|
||||
Not all sections are required for every tutorial, but this is the standard flow.
|
||||
### Tutorial Checklist
|
||||
|
||||
- [ ] Hook describes outcome in 1-2 sentences
|
||||
- [ ] "What You'll Learn" section present
|
||||
- [ ] Prerequisites in admonition
|
||||
- [ ] Quick Path TL;DR admonition at top
|
||||
- [ ] Tables for phases, commands, agents
|
||||
- [ ] "What You've Accomplished" section present
|
||||
- [ ] Quick Reference table present
|
||||
- [ ] Common Questions section present
|
||||
- [ ] Getting Help section present
|
||||
- [ ] Key Takeaways admonition at end
|
||||
|
||||
## How-To Structure
|
||||
|
||||
How-to guides are task-focused and shorter than tutorials. They answer "How do I do X?" for users who already understand the basics.
|
||||
|
||||
```
|
||||
```text
|
||||
1. Title + Hook (one sentence: "Use the `X` workflow to...")
|
||||
2. When to Use This (bullet list of scenarios)
|
||||
3. When to Skip This (optional - for workflows that aren't always needed)
|
||||
3. When to Skip This (optional)
|
||||
4. Prerequisites (note admonition)
|
||||
5. Steps (numbered ### subsections)
|
||||
6. What You Get (output/artifacts produced)
|
||||
7. Example (optional - concrete usage scenario)
|
||||
8. Tips (optional - best practices, common pitfalls)
|
||||
9. Next Steps (optional - what to do after completion)
|
||||
7. Example (optional)
|
||||
8. Tips (optional)
|
||||
9. Next Steps (optional)
|
||||
```
|
||||
|
||||
Include sections only when they add value. A simple how-to might only need Hook, Prerequisites, Steps, and What You Get.
|
||||
|
||||
### How-To vs Tutorial
|
||||
|
||||
| Aspect | How-To | Tutorial |
|
||||
|--------|--------|----------|
|
||||
| **Length** | 50-150 lines | 200-400 lines |
|
||||
| **Audience** | Users who know the basics | New users learning concepts |
|
||||
| **Focus** | Complete a specific task | Understand a workflow end-to-end |
|
||||
| **Sections** | 5-8 sections | 12-15 sections |
|
||||
| **Examples** | Brief, inline | Detailed, step-by-step |
|
||||
|
||||
### How-To Visual Elements
|
||||
|
||||
Use admonitions strategically in how-to guides:
|
||||
|
||||
| Admonition | Use In How-To |
|
||||
|------------|---------------|
|
||||
| `:::note[Prerequisites]` | Required dependencies, agents, prior steps |
|
||||
| `:::tip[Pro Tip]` | Optional shortcuts or best practices |
|
||||
| `:::caution[Common Mistake]` | Pitfalls to avoid |
|
||||
| `:::note[Example]` | Brief usage example inline with steps |
|
||||
|
||||
**Guidelines:**
|
||||
- **1-2 admonitions max** per how-to (they're shorter than tutorials)
|
||||
- **Prerequisites as admonition** makes scanning easier
|
||||
- **Tips section** can be a flat list instead of admonition if there are multiple tips
|
||||
- **Skip admonitions entirely** for very simple how-tos
|
||||
|
||||
### How-To Checklist
|
||||
|
||||
Before submitting a how-to:
|
||||
|
||||
- [ ] Hook is one clear sentence starting with "Use the `X` workflow to..."
|
||||
- [ ] When to Use This has 3-5 bullet points
|
||||
- [ ] Prerequisites listed (admonition or flat list)
|
||||
- [ ] Hook starts with "Use the `X` workflow to..."
|
||||
- [ ] "When to Use This" has 3-5 bullet points
|
||||
- [ ] Prerequisites listed
|
||||
- [ ] Steps are numbered `###` subsections with action verbs
|
||||
- [ ] What You Get describes output artifacts
|
||||
- [ ] No horizontal rules (`---`)
|
||||
- [ ] No `####` headers
|
||||
- [ ] No "Related" section (sidebar handles navigation)
|
||||
- [ ] 1-2 admonitions maximum
|
||||
- [ ] "What You Get" describes output artifacts
|
||||
|
||||
## Explanation Structure
|
||||
|
||||
Explanation documents help users understand concepts, features, and design decisions. They answer "What is X?" and "Why does X matter?" rather than "How do I do X?"
|
||||
### Types
|
||||
|
||||
### Types of Explanation Documents
|
||||
| Type | Example |
|
||||
|------|---------|
|
||||
| **Index/Landing** | `core-concepts/index.md` |
|
||||
| **Concept** | `what-are-agents.md` |
|
||||
| **Feature** | `quick-flow.md` |
|
||||
| **Philosophy** | `why-solutioning-matters.md` |
|
||||
| **FAQ** | `brownfield-faq.md` |
|
||||
|
||||
| Type | Purpose | Example |
|
||||
|------|---------|---------|
|
||||
| **Index/Landing** | Overview of a topic area with navigation | `core-concepts/index.md` |
|
||||
| **Concept** | Define and explain a core concept | `what-are-agents.md` |
|
||||
| **Feature** | Deep dive into a specific capability | `quick-flow.md` |
|
||||
| **Philosophy** | Explain design decisions and rationale | `why-solutioning-matters.md` |
|
||||
| **FAQ** | Answer common questions (see FAQ Sections below) | `brownfield-faq.md` |
|
||||
### General Template
|
||||
|
||||
### General Explanation Structure
|
||||
|
||||
```
|
||||
1. Title + Hook (1-2 sentences explaining the topic)
|
||||
```text
|
||||
1. Title + Hook (1-2 sentences)
|
||||
2. Overview/Definition (what it is, why it matters)
|
||||
3. Key Concepts (### subsections for main ideas)
|
||||
4. Comparison Table (optional - when comparing options)
|
||||
5. When to Use / When Not to Use (optional - decision guidance)
|
||||
6. Diagram (optional - mermaid for processes/flows)
|
||||
7. Next Steps (optional - where to go from here)
|
||||
3. Key Concepts (### subsections)
|
||||
4. Comparison Table (optional)
|
||||
5. When to Use / When Not to Use (optional)
|
||||
6. Diagram (optional - mermaid, 1 per doc max)
|
||||
7. Next Steps (optional)
|
||||
```
|
||||
|
||||
### Index/Landing Pages
|
||||
|
||||
Index pages orient users within a topic area.
|
||||
|
||||
```
|
||||
1. Title + Hook (one sentence overview)
|
||||
```text
|
||||
1. Title + Hook (one sentence)
|
||||
2. Content Table (links with descriptions)
|
||||
3. Getting Started (numbered list for new users)
|
||||
4. Choose Your Path (optional - decision tree for different goals)
|
||||
3. Getting Started (numbered list)
|
||||
4. Choose Your Path (optional - decision tree)
|
||||
```
|
||||
|
||||
**Example hook:** "Understanding the fundamental building blocks of the BMad Method."
|
||||
|
||||
### Concept Explainers
|
||||
|
||||
Concept pages define and explain core ideas.
|
||||
|
||||
```text
|
||||
1. Title + Hook (what it is)
|
||||
2. Types/Categories (### subsections) (optional)
|
||||
3. Key Differences Table
|
||||
4. Components/Parts
|
||||
5. Which Should You Use?
|
||||
6. Creating/Customizing (pointer to how-to guides)
|
||||
```
|
||||
1. Title + Hook (what it is in one sentence)
|
||||
2. Types/Categories (if applicable, with ### subsections)
|
||||
3. Key Differences Table (comparing types/options)
|
||||
4. Components/Parts (breakdown of elements)
|
||||
5. Which Should You Use? (decision guidance)
|
||||
6. Creating/Customizing (brief pointer to how-to guides)
|
||||
```
|
||||
|
||||
**Example hook:** "Agents are AI assistants that help you accomplish tasks. Each agent has a unique personality, specialized capabilities, and an interactive menu."
|
||||
|
||||
### Feature Explainers
|
||||
|
||||
Feature pages provide deep dives into specific capabilities.
|
||||
|
||||
```
|
||||
1. Title + Hook (what the feature does)
|
||||
```text
|
||||
1. Title + Hook (what it does)
|
||||
2. Quick Facts (optional - "Perfect for:", "Time to:")
|
||||
3. When to Use / When Not to Use (with bullet lists)
|
||||
4. How It Works (process overview, mermaid diagram if helpful)
|
||||
5. Key Benefits (what makes it valuable)
|
||||
6. Comparison Table (vs alternatives if applicable)
|
||||
7. When to Graduate/Upgrade (optional - when to use something else)
|
||||
3. When to Use / When Not to Use
|
||||
4. How It Works (mermaid diagram optional)
|
||||
5. Key Benefits
|
||||
6. Comparison Table (optional)
|
||||
7. When to Graduate/Upgrade (optional)
|
||||
```
|
||||
|
||||
**Example hook:** "Quick Spec Flow is a streamlined alternative to the full BMad Method for Quick Flow track projects."
|
||||
|
||||
### Philosophy/Rationale Documents
|
||||
|
||||
Philosophy pages explain design decisions and reasoning.
|
||||
|
||||
```text
|
||||
1. Title + Hook (the principle)
|
||||
2. The Problem
|
||||
3. The Solution
|
||||
4. Key Principles (### subsections)
|
||||
5. Benefits
|
||||
6. When This Applies
|
||||
```
|
||||
1. Title + Hook (the principle or decision)
|
||||
2. The Problem (what issue this addresses)
|
||||
3. The Solution (how this approach solves it)
|
||||
4. Key Principles (### subsections for main ideas)
|
||||
5. Benefits (what users gain)
|
||||
6. When This Applies (scope of the principle)
|
||||
```
|
||||
|
||||
**Example hook:** "Phase 3 (Solutioning) translates **what** to build (from Planning) into **how** to build it (technical design)."
|
||||
|
||||
### Explanation Visual Elements
|
||||
|
||||
Use these elements strategically in explanation documents:
|
||||
|
||||
| Element | Use For |
|
||||
|---------|---------|
|
||||
| **Comparison tables** | Contrasting types, options, or approaches |
|
||||
| **Mermaid diagrams** | Process flows, phase sequences, decision trees |
|
||||
| **"Best for:" lists** | Quick decision guidance |
|
||||
| **Code examples** | Illustrating concepts (keep brief) |
|
||||
|
||||
**Guidelines:**
|
||||
- **Use diagrams sparingly** — one mermaid diagram per document maximum
|
||||
- **Tables over prose** — for any comparison of 3+ items
|
||||
- **Avoid step-by-step instructions** — point to how-to guides instead
|
||||
|
||||
### Explanation Checklist
|
||||
|
||||
Before submitting an explanation document:
|
||||
|
||||
- [ ] Hook clearly states what the document explains
|
||||
- [ ] Content organized into scannable `##` sections
|
||||
- [ ] Comparison tables used for contrasting options
|
||||
- [ ] No horizontal rules (`---`)
|
||||
- [ ] No `####` headers
|
||||
- [ ] No "Related" section (sidebar handles navigation)
|
||||
- [ ] No "Next:" navigation links (sidebar handles navigation)
|
||||
- [ ] Diagrams have clear labels and flow
|
||||
- [ ] Links to how-to guides for "how do I do this?" questions
|
||||
- [ ] 2-3 admonitions maximum
|
||||
- [ ] Hook states what document explains
|
||||
- [ ] Content in scannable `##` sections
|
||||
- [ ] Comparison tables for 3+ options
|
||||
- [ ] Diagrams have clear labels
|
||||
- [ ] Links to how-to guides for procedural questions
|
||||
- [ ] 2-3 admonitions max per document
|
||||
|
||||
## Reference Structure
|
||||
|
||||
Reference documents provide quick lookup information for users who know what they're looking for. They answer "What are the options?" and "What does X do?" rather than explaining concepts or teaching skills.
|
||||
### Types
|
||||
|
||||
### Types of Reference Documents
|
||||
|
||||
| Type | Purpose | Example |
|
||||
|------|---------|---------|
|
||||
| **Index/Landing** | Navigation to reference content | `workflows/index.md` |
|
||||
| **Catalog** | Quick-reference list of items | `agents/index.md` |
|
||||
| **Deep-Dive** | Detailed single-item reference | `document-project.md` |
|
||||
| **Configuration** | Settings and config documentation | `core-tasks.md` |
|
||||
| **Glossary** | Term definitions | `glossary/index.md` |
|
||||
| **Comprehensive** | Extensive multi-item reference | `bmgd-workflows.md` |
|
||||
| Type | Example |
|
||||
|------|---------|
|
||||
| **Index/Landing** | `workflows/index.md` |
|
||||
| **Catalog** | `agents/index.md` |
|
||||
| **Deep-Dive** | `document-project.md` |
|
||||
| **Configuration** | `core-tasks.md` |
|
||||
| **Glossary** | `glossary/index.md` |
|
||||
| **Comprehensive** | `bmgd-workflows.md` |
|
||||
|
||||
### Reference Index Pages
|
||||
|
||||
For navigation landing pages:
|
||||
|
||||
```
|
||||
1. Title + Hook (one sentence describing scope)
|
||||
2. Content Sections (## for each category)
|
||||
- Bullet list with links and brief descriptions
|
||||
```
|
||||
|
||||
Keep these minimal — their job is navigation, not explanation.
|
||||
|
||||
### Catalog Reference (Item Lists)
|
||||
|
||||
For quick-reference lists of items:
|
||||
|
||||
```
|
||||
```text
|
||||
1. Title + Hook (one sentence)
|
||||
2. Content Sections (## for each category)
|
||||
- Bullet list with links and descriptions
|
||||
```
|
||||
|
||||
### Catalog Reference
|
||||
|
||||
```text
|
||||
1. Title + Hook
|
||||
2. Items (## for each item)
|
||||
- Brief description (one sentence)
|
||||
- **Commands:** or **Key Info:** as flat list
|
||||
3. Universal/Shared (## section if applicable)
|
||||
3. Universal/Shared (## section) (optional)
|
||||
```
|
||||
|
||||
**Guidelines:**
|
||||
- Use `##` for items, not `###`
|
||||
- No horizontal rules between items — whitespace is sufficient
|
||||
- No "Related" section — sidebar handles navigation
|
||||
- Keep descriptions to 1 sentence per item
|
||||
|
||||
### Item Deep-Dive Reference
|
||||
|
||||
For detailed single-item documentation:
|
||||
|
||||
```
|
||||
```text
|
||||
1. Title + Hook (one sentence purpose)
|
||||
2. Quick Facts (optional note admonition)
|
||||
- Module, Command, Input, Output as list
|
||||
3. Purpose/Overview (## section)
|
||||
4. How to Invoke (code block)
|
||||
5. Key Sections (## for each major aspect)
|
||||
- Use ### for sub-options within sections
|
||||
5. Key Sections (## for each aspect)
|
||||
- Use ### for sub-options
|
||||
6. Notes/Caveats (tip or caution admonition)
|
||||
```
|
||||
|
||||
**Guidelines:**
|
||||
- Start with "quick facts" so readers immediately know scope
|
||||
- Use admonitions for important caveats
|
||||
- No "Related Documentation" section — sidebar handles this
|
||||
|
||||
### Configuration Reference
|
||||
|
||||
For settings, tasks, and config documentation:
|
||||
|
||||
```
|
||||
1. Title + Hook (one sentence explaining what these configure)
|
||||
```text
|
||||
1. Title + Hook
|
||||
2. Table of Contents (jump links if 4+ items)
|
||||
3. Items (## for each config/task)
|
||||
- **Bold summary** — one sentence describing what it does
|
||||
- **Use it when:** bullet list of scenarios
|
||||
- **How it works:** numbered steps
|
||||
- **Output:** expected result (if applicable)
|
||||
- **Bold summary** — one sentence
|
||||
- **Use it when:** bullet list
|
||||
- **How it works:** numbered steps (3-5 max)
|
||||
- **Output:** expected result (optional)
|
||||
```
|
||||
|
||||
**Guidelines:**
|
||||
- Table of contents only needed for 4+ items
|
||||
- Keep "How it works" to 3-5 steps maximum
|
||||
- No horizontal rules between items
|
||||
|
||||
### Glossary Reference
|
||||
|
||||
For term definitions:
|
||||
|
||||
```
|
||||
1. Title + Hook (one sentence)
|
||||
2. Navigation (jump links to categories)
|
||||
3. Categories (## for each category)
|
||||
- Terms (### for each term)
|
||||
- Definition (1-3 sentences, no prefix)
|
||||
- Related context or example (optional)
|
||||
```
|
||||
|
||||
**Guidelines:**
|
||||
- Group related terms into categories
|
||||
- Keep definitions concise — link to explanation docs for depth
|
||||
- Use `###` for terms (makes them linkable and scannable)
|
||||
- No horizontal rules between terms
|
||||
|
||||
### Comprehensive Reference Guide
|
||||
|
||||
For extensive multi-item references:
|
||||
|
||||
```
|
||||
1. Title + Hook (one sentence)
|
||||
```text
|
||||
1. Title + Hook
|
||||
2. Overview (## section)
|
||||
- Diagram or table showing organization
|
||||
3. Major Sections (## for each phase/category)
|
||||
- Items (### for each item)
|
||||
- Standardized fields: Command, Agent, Input, Output, Description
|
||||
- Optional: Steps, Features, Use when
|
||||
4. Next Steps (optional — only if genuinely helpful)
|
||||
4. Next Steps (optional)
|
||||
```
|
||||
|
||||
**Guidelines:**
|
||||
- Standardize item fields across all items in the guide
|
||||
- Use tables for comparing multiple items at once
|
||||
- One diagram maximum per document
|
||||
- No horizontal rules — use `##` sections for separation
|
||||
|
||||
### General Reference Guidelines
|
||||
|
||||
These apply to all reference documents:
|
||||
|
||||
| Do | Don't |
|
||||
|----|-------|
|
||||
| Use `##` for major sections, `###` for items within | Use `####` headers |
|
||||
| Use whitespace for separation | Use horizontal rules (`---`) |
|
||||
| Link to explanation docs for "why" | Explain concepts inline |
|
||||
| Use tables for structured data | Use nested lists |
|
||||
| Use admonitions for important notes | Use bold paragraphs for callouts |
|
||||
| Keep descriptions to 1-2 sentences | Write paragraphs of explanation |
|
||||
|
||||
### Reference Admonitions
|
||||
|
||||
Use sparingly — 1-2 maximum per reference document:
|
||||
|
||||
| Admonition | Use In Reference |
|
||||
|------------|------------------|
|
||||
| `:::note[Prerequisites]` | Dependencies needed before using |
|
||||
| `:::tip[Pro Tip]` | Shortcuts or advanced usage |
|
||||
| `:::caution[Important]` | Critical caveats or warnings |
|
||||
|
||||
### Reference Checklist
|
||||
|
||||
Before submitting a reference document:
|
||||
|
||||
- [ ] Hook clearly states what the document references
|
||||
- [ ] Appropriate structure for reference type (catalog, deep-dive, etc.)
|
||||
- [ ] No horizontal rules (`---`)
|
||||
- [ ] No `####` headers
|
||||
- [ ] No "Related" section (sidebar handles navigation)
|
||||
- [ ] Hook states what document references
|
||||
- [ ] Structure matches reference type
|
||||
- [ ] Items use consistent structure throughout
|
||||
- [ ] Descriptions are 1-2 sentences maximum
|
||||
- [ ] Tables used for structured/comparative data
|
||||
- [ ] 1-2 admonitions maximum
|
||||
- [ ] Tables for structured/comparative data
|
||||
- [ ] Links to explanation docs for conceptual depth
|
||||
- [ ] 1-2 admonitions max
|
||||
|
||||
## Glossary Structure
|
||||
|
||||
Glossaries provide quick-reference definitions for project terminology. Unlike other reference documents, glossaries prioritize compact scanability over narrative explanation.
|
||||
Starlight generates right-side "On this page" navigation from headers:
|
||||
|
||||
### Layout Strategy
|
||||
|
||||
Starlight auto-generates a right-side "On this page" navigation from headers. Use this to your advantage:
|
||||
|
||||
- **Categories as `##` headers** — Appear in right nav for quick jumping
|
||||
- **Terms in tables** — Compact rows, not individual headers
|
||||
- **No inline TOC** — Right sidebar handles navigation; inline TOC is redundant
|
||||
- **Right nav shows categories only** — Cleaner than listing every term
|
||||
|
||||
This approach reduces content length by ~70% while improving navigation.
|
||||
- Categories as `##` headers — appear in right nav
|
||||
- Terms in tables — compact rows, not individual headers
|
||||
- No inline TOC — right sidebar handles navigation
|
||||
|
||||
### Table Format
|
||||
|
||||
Each category uses a two-column table:
|
||||
|
||||
```md
|
||||
## Category Name
|
||||
|
||||
|
|
@ -421,250 +309,35 @@ Each category uses a two-column table:
|
|||
| **Workflow** | Multi-step guided process that orchestrates AI agent activities to produce deliverables. |
|
||||
```
|
||||
|
||||
### Definition Guidelines
|
||||
### Definition Rules
|
||||
|
||||
| Do | Don't |
|
||||
|----|-------|
|
||||
| Start with what it IS or DOES | Start with "This is..." or "A [term] is..." |
|
||||
| Keep to 1-2 sentences | Write multi-paragraph explanations |
|
||||
| Bold the term name in the cell | Use plain text for terms |
|
||||
| Link to docs for deep dives | Explain full concepts inline |
|
||||
| Bold term name in cell | Use plain text for terms |
|
||||
|
||||
### Context Markers
|
||||
|
||||
For terms with limited scope, add italic context at the start of the definition:
|
||||
Add italic context at definition start for limited-scope terms:
|
||||
|
||||
```md
|
||||
| **Tech-Spec** | *Quick Flow only.* Comprehensive technical plan for small changes. |
|
||||
| **PRD** | *BMad Method/Enterprise.* Product-level planning document with vision and goals. |
|
||||
```
|
||||
|
||||
Standard markers:
|
||||
- `*Quick Flow only.*`
|
||||
- `*BMad Method/Enterprise.*`
|
||||
- `*Phase N.*`
|
||||
- `*BMGD.*`
|
||||
- `*Brownfield.*`
|
||||
|
||||
### Cross-References
|
||||
|
||||
Link related terms when helpful. Reference the category anchor since individual terms aren't headers:
|
||||
|
||||
```md
|
||||
| **Tech-Spec** | *Quick Flow only.* Technical plan for small changes. See [PRD](#planning-documents). |
|
||||
```
|
||||
|
||||
### Organization
|
||||
|
||||
- **Alphabetize terms** within each category table
|
||||
- **Alphabetize categories** or order by logical progression (foundational → specific)
|
||||
- **No catch-all sections** — Every term belongs in a specific category
|
||||
|
||||
### Glossary Checklist
|
||||
|
||||
Before submitting glossary changes:
|
||||
|
||||
- [ ] Terms in tables, not individual headers
|
||||
- [ ] Terms alphabetized within each category
|
||||
- [ ] No inline TOC (right nav handles navigation)
|
||||
- [ ] No horizontal rules (`---`)
|
||||
- [ ] Definitions are 1-2 sentences
|
||||
- [ ] Context markers italicized at definition start
|
||||
- [ ] Term names bolded in table cells
|
||||
- [ ] Terms alphabetized within categories
|
||||
- [ ] Definitions 1-2 sentences
|
||||
- [ ] Context markers italicized
|
||||
- [ ] Term names bolded in cells
|
||||
- [ ] No "A [term] is..." definitions
|
||||
|
||||
## Visual Hierarchy
|
||||
|
||||
### Avoid
|
||||
|
||||
| Pattern | Problem |
|
||||
|---------|---------|
|
||||
| `---` horizontal rules | Fragment the reading flow |
|
||||
| `####` deep headers | Create visual noise |
|
||||
| **Important:** bold paragraphs | Blend into body text |
|
||||
| Deeply nested lists | Hard to scan |
|
||||
| Code blocks for non-code | Confusing semantics |
|
||||
|
||||
### Use Instead
|
||||
|
||||
| Pattern | When to Use |
|
||||
|---------|-------------|
|
||||
| White space + section headers | Natural content separation |
|
||||
| Bold text within paragraphs | Inline emphasis |
|
||||
| Admonitions | Callouts that need attention |
|
||||
| Tables | Structured comparisons |
|
||||
| Flat lists | Scannable options |
|
||||
|
||||
## Admonitions
|
||||
|
||||
Use Starlight admonitions strategically:
|
||||
|
||||
```md
|
||||
:::tip[Title]
|
||||
Shortcuts, best practices, "pro tips"
|
||||
:::
|
||||
|
||||
:::note[Title]
|
||||
Context, definitions, examples, prerequisites
|
||||
:::
|
||||
|
||||
:::caution[Title]
|
||||
Caveats, potential issues, things to watch out for
|
||||
:::
|
||||
|
||||
:::danger[Title]
|
||||
Critical warnings only — data loss, security issues
|
||||
:::
|
||||
```
|
||||
|
||||
### Standard Admonition Uses
|
||||
|
||||
| Admonition | Standard Use in Tutorials |
|
||||
|------------|---------------------------|
|
||||
| `:::note[Prerequisites]` | What users need before starting |
|
||||
| `:::tip[Quick Path]` | TL;DR summary at top of tutorial |
|
||||
| `:::caution[Fresh Chats]` | Context limitation reminders |
|
||||
| `:::note[Example]` | Command/response examples |
|
||||
| `:::tip[Check Your Status]` | How to verify progress |
|
||||
| `:::tip[Remember These]` | Key takeaways at end |
|
||||
|
||||
### Admonition Guidelines
|
||||
|
||||
- **Always include a title** for tip, info, and warning
|
||||
- **Keep content brief** — 1-3 sentences ideal
|
||||
- **Don't overuse** — More than 3-4 per major section feels noisy
|
||||
- **Don't nest** — Admonitions inside admonitions are hard to read
|
||||
|
||||
## Headers
|
||||
|
||||
### Budget
|
||||
|
||||
- **8-12 `##` sections** for full tutorials following standard structure
|
||||
- **2-3 `###` subsections** per `##` section maximum
|
||||
- **Avoid `####` entirely** — use bold text or admonitions instead
|
||||
|
||||
### Naming
|
||||
|
||||
- Use action verbs for steps: "Install BMad", "Create Your Plan"
|
||||
- Use nouns for reference sections: "Common Questions", "Quick Reference"
|
||||
- Keep headers short and scannable
|
||||
|
||||
## Code Blocks
|
||||
|
||||
### Do
|
||||
|
||||
```md
|
||||
```bash
|
||||
npx bmad-method install
|
||||
```
|
||||
```
|
||||
|
||||
### Don't
|
||||
|
||||
````md
|
||||
```
|
||||
You: Do something
|
||||
Agent: [Response here]
|
||||
```
|
||||
````
|
||||
|
||||
For command/response examples, use an admonition instead:
|
||||
|
||||
```md
|
||||
:::note[Example]
|
||||
Run `workflow-status` and the agent will tell you the next recommended workflow.
|
||||
:::
|
||||
```
|
||||
|
||||
## Tables
|
||||
|
||||
Use tables for:
|
||||
- Phases and what happens in each
|
||||
- Agent roles and when to use them
|
||||
- Command references
|
||||
- Comparing options
|
||||
- Step sequences with multiple attributes
|
||||
|
||||
Keep tables simple:
|
||||
- 2-4 columns maximum
|
||||
- Short cell content
|
||||
- Left-align text, right-align numbers
|
||||
|
||||
### Standard Tables
|
||||
|
||||
**Phases Table:**
|
||||
```md
|
||||
| Phase | Name | What Happens |
|
||||
|-------|------|--------------|
|
||||
| 1 | Analysis | Brainstorm, research *(optional)* |
|
||||
| 2 | Planning | Requirements — PRD or tech-spec *(required)* |
|
||||
```
|
||||
|
||||
**Quick Reference Table:**
|
||||
```md
|
||||
| Command | Agent | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `*workflow-init` | Analyst | Initialize a new project |
|
||||
| `*prd` | PM | Create Product Requirements Document |
|
||||
```
|
||||
|
||||
**Build Cycle Table:**
|
||||
```md
|
||||
| Step | Agent | Workflow | Purpose |
|
||||
|------|-------|----------|---------|
|
||||
| 1 | SM | `create-story` | Create story file from epic |
|
||||
| 2 | DEV | `dev-story` | Implement the story |
|
||||
```
|
||||
|
||||
## Lists
|
||||
|
||||
### Flat Lists (Preferred)
|
||||
|
||||
```md
|
||||
- **Option A** — Description of option A
|
||||
- **Option B** — Description of option B
|
||||
- **Option C** — Description of option C
|
||||
```
|
||||
|
||||
### Numbered Steps
|
||||
|
||||
```md
|
||||
1. Load the **PM agent** in a new chat
|
||||
2. Run the PRD workflow: `*prd`
|
||||
3. Output: `PRD.md`
|
||||
```
|
||||
|
||||
### Avoid Deep Nesting
|
||||
|
||||
```md
|
||||
<!-- Don't do this -->
|
||||
1. First step
|
||||
- Sub-step A
|
||||
- Detail 1
|
||||
- Detail 2
|
||||
- Sub-step B
|
||||
2. Second step
|
||||
```
|
||||
|
||||
Instead, break into separate sections or use an admonition for context.
|
||||
|
||||
## Links
|
||||
|
||||
- Use descriptive link text: `[Tutorial Style Guide](./tutorial-style.md)`
|
||||
- Avoid "click here" or bare URLs
|
||||
- Prefer relative paths within docs
|
||||
|
||||
## Images
|
||||
|
||||
- Always include alt text
|
||||
- Add a caption in italics below: `*Description of the image.*`
|
||||
- Use SVG for diagrams when possible
|
||||
- Store in `./images/` relative to the document
|
||||
|
||||
## FAQ Sections
|
||||
|
||||
Use a TOC with jump links, `###` headers for questions, and direct answers:
|
||||
|
||||
```md
|
||||
## Questions
|
||||
|
||||
|
|
@ -679,88 +352,16 @@ Only for BMad Method and Enterprise tracks. Quick Flow skips to implementation.
|
|||
|
||||
Yes. The SM agent has a `correct-course` workflow for handling scope changes.
|
||||
|
||||
**Have a question not answered here?** Please [open an issue](...) or ask in [Discord](...) so we can add it!
|
||||
**Have a question not answered here?** [Open an issue](...) or ask in [Discord](...).
|
||||
```
|
||||
|
||||
### FAQ Guidelines
|
||||
## Validation Commands
|
||||
|
||||
- **TOC at top** — Jump links under `## Questions` for quick navigation
|
||||
- **`###` headers** — Questions are scannable and linkable (no `Q:` prefix)
|
||||
- **Direct answers** — No `**A:**` prefix, just the answer
|
||||
- **No "Related Documentation"** — Sidebar handles navigation; avoid repetitive links
|
||||
- **End with CTA** — "Have a question not answered here?" with issue/Discord links
|
||||
|
||||
## Folder Structure Blocks
|
||||
|
||||
Show project structure in "What You've Accomplished":
|
||||
|
||||
````md
|
||||
Your project now has:
|
||||
Before submitting documentation changes:
|
||||
|
||||
```bash
|
||||
npm run docs:fix-links # Preview link format fixes
|
||||
npm run docs:fix-links -- --write # Apply fixes
|
||||
npm run docs:validate-links # Check links exist
|
||||
npm run docs:build # Verify no build errors
|
||||
```
|
||||
your-project/
|
||||
├── _bmad/ # BMad configuration
|
||||
├── _bmad-output/
|
||||
│ ├── PRD.md # Your requirements document
|
||||
│ └── bmm-workflow-status.yaml # Progress tracking
|
||||
└── ...
|
||||
```
|
||||
````
|
||||
|
||||
## Example: Before and After
|
||||
|
||||
### Before (Noisy)
|
||||
|
||||
```md
|
||||
---
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Step 1: Initialize
|
||||
|
||||
#### What happens during init?
|
||||
|
||||
**Important:** You need to describe your project.
|
||||
|
||||
1. Your project goals
|
||||
- What you want to build
|
||||
- Why you're building it
|
||||
2. The complexity
|
||||
- Small, medium, or large
|
||||
|
||||
---
|
||||
```
|
||||
|
||||
### After (Clean)
|
||||
|
||||
```md
|
||||
## Step 1: Initialize Your Project
|
||||
|
||||
Load the **Analyst agent** in your IDE, wait for the menu, then run `workflow-init`.
|
||||
|
||||
:::note[What Happens]
|
||||
You'll describe your project goals and complexity. The workflow then recommends a planning track.
|
||||
:::
|
||||
```
|
||||
|
||||
## Checklist
|
||||
|
||||
Before submitting a tutorial:
|
||||
|
||||
- [ ] Follows the standard structure
|
||||
- [ ] Has version/module notice if applicable
|
||||
- [ ] Has "What You'll Learn" section
|
||||
- [ ] Has Prerequisites admonition
|
||||
- [ ] Has Quick Path TL;DR admonition
|
||||
- [ ] No horizontal rules (`---`)
|
||||
- [ ] No `####` headers
|
||||
- [ ] Admonitions used for callouts (not bold paragraphs)
|
||||
- [ ] Tables used for structured data (phases, commands, agents)
|
||||
- [ ] Lists are flat (no deep nesting)
|
||||
- [ ] Has "What You've Accomplished" section
|
||||
- [ ] Has Quick Reference table
|
||||
- [ ] Has Common Questions section
|
||||
- [ ] Has Getting Help section
|
||||
- [ ] Has Key Takeaways admonition
|
||||
- [ ] All links use descriptive text
|
||||
- [ ] Images have alt text and captions
|
||||
|
|
|
|||
|
|
@ -7,11 +7,10 @@ Comprehensive guides to BMad's AI agents — their roles, capabilities, and how
|
|||
|
||||
## Agent Guides
|
||||
|
||||
| Agent | Description |
|
||||
|-------|-------------|
|
||||
| **[Agent Roles](/docs/explanation/core-concepts/agent-roles.md)** | Overview of all BMM agent roles and responsibilities |
|
||||
| **[Quick Flow Solo Dev (Barry)](/docs/explanation/agents/barry-quick-flow.md)** | The dedicated agent for rapid development |
|
||||
| **[Game Development Agents](/docs/explanation/game-dev/agents.md)** | Complete guide to BMGD's specialized game dev agents |
|
||||
| Agent | Description |
|
||||
| ------------------------------------------------------------------------------- | ---------------------------------------------------- |
|
||||
| **[Agent Roles](/docs/explanation/core-concepts/agent-roles.md)** | Overview of all BMM agent roles and responsibilities |
|
||||
| **[Quick Flow Solo Dev (Barry)](/docs/explanation/agents/barry-quick-flow.md)** | The dedicated agent for rapid development |
|
||||
|
||||
## Getting Started
|
||||
|
||||
|
|
|
|||
|
|
@ -20,10 +20,13 @@ This flexibility enables:
|
|||
|
||||
## Categories
|
||||
|
||||
- [Categories](#categories)
|
||||
- [Custom Stand-Alone Modules](#custom-stand-alone-modules)
|
||||
- [Custom Add-On Modules](#custom-add-on-modules)
|
||||
- [Custom Global Modules](#custom-global-modules)
|
||||
- [Custom Agents](#custom-agents)
|
||||
- [BMad Tiny Agents](#bmad-tiny-agents)
|
||||
- [Simple and Expert Agents](#simple-and-expert-agents)
|
||||
- [Custom Workflows](#custom-workflows)
|
||||
|
||||
## Custom Stand-Alone Modules
|
||||
|
|
@ -59,7 +62,6 @@ Similar to Custom Stand-Alone Modules, but designed to add functionality that ap
|
|||
|
||||
Examples include:
|
||||
|
||||
- The current TTS (Text-to-Speech) functionality for Claude, which will soon be converted to a global module
|
||||
- The core module, which is always installed and provides all agents with party mode and advanced elicitation capabilities
|
||||
- Installation and update tools that work with any BMad method configuration
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ A module is a self-contained package that includes:
|
|||
- **Configuration** - Module-specific settings
|
||||
- **Documentation** - Usage guides and reference
|
||||
|
||||
## Official Modules
|
||||
## Official BMad Method and Builder Modules
|
||||
|
||||
:::note[Core is Always Installed]
|
||||
The Core module is automatically included with every BMad installation. It provides the foundation that other modules build upon.
|
||||
|
|
@ -37,17 +37,24 @@ Create custom solutions:
|
|||
- Workflow authoring tools
|
||||
- Module scaffolding
|
||||
|
||||
## Additional Official BMad Modules
|
||||
|
||||
These are officially maintained modules by BMad but have their own repo's and docs.
|
||||
These give a good idea also of what can be done with the BMad builder and creating your own custom modules.
|
||||
|
||||
### Creative Intelligence Suite (CIS)
|
||||
Innovation and creativity:
|
||||
- Creative thinking techniques
|
||||
- Innovation strategy workflows
|
||||
- Storytelling and ideation
|
||||
- [Available Here](https://github.com/bmad-code-org/bmad-module-creative-intelligence-suite)
|
||||
|
||||
### BMad Game Dev (BMGD)
|
||||
Game development specialization:
|
||||
- Game design workflows
|
||||
- Narrative development
|
||||
- Performance testing frameworks
|
||||
- [Available Here](https://github.com/bmad-code-org/bmad-module-game-dev-studio)
|
||||
|
||||
## Module Structure
|
||||
|
||||
|
|
|
|||
|
|
@ -163,7 +163,7 @@ Before building a workflow, answer these questions:
|
|||
|
||||
The best way to understand workflows is to study real examples. Look at the official BMad modules:
|
||||
|
||||
- **BMB (Module Builder)**: Workflow and agent creation workflows
|
||||
- **BMB (Module Builder)**: Module, Workflow and Agent creation workflows
|
||||
- **BMM (Business Method Module)**: Complete software development pipeline from brainstorming through sprint planning
|
||||
- **BMGD (Game Development Module)**: Game design briefs, narratives, architecture
|
||||
- **CIS (Creativity, Innovation, Strategy)**: Brainstorming, design thinking, storytelling, innovation strategy
|
||||
|
|
|
|||
|
|
@ -1,103 +0,0 @@
|
|||
---
|
||||
title: "Creative Intelligence Suite (CIS)"
|
||||
description: AI-powered creative facilitation with the Creative Intelligence Suite
|
||||
---
|
||||
|
||||
AI-powered creative facilitation transforming strategic thinking through expert coaching across five specialized domains.
|
||||
|
||||
## Core Capabilities
|
||||
|
||||
CIS provides structured creative methodologies through distinctive agent personas who act as master facilitators, drawing out insights through strategic questioning rather than generating solutions directly.
|
||||
|
||||
## Specialized Agents
|
||||
|
||||
- **Carson** - Brainstorming Specialist (energetic facilitator)
|
||||
- **Maya** - Design Thinking Maestro (jazz-like improviser)
|
||||
- **Dr. Quinn** - Problem Solver (detective-scientist hybrid)
|
||||
- **Victor** - Innovation Oracle (bold strategic precision)
|
||||
- **Sophia** - Master Storyteller (whimsical narrator)
|
||||
|
||||
## Interactive Workflows
|
||||
|
||||
**5 Workflows** with **150+ Creative Techniques:**
|
||||
|
||||
### Brainstorming
|
||||
|
||||
36 techniques across 7 categories for ideation:
|
||||
- Divergent/convergent thinking
|
||||
- Lateral connections
|
||||
- Forced associations
|
||||
|
||||
### Design Thinking
|
||||
|
||||
Complete 5-phase human-centered process:
|
||||
- Empathize → Define → Ideate → Prototype → Test
|
||||
- User journey mapping
|
||||
- Rapid iteration
|
||||
|
||||
### Problem Solving
|
||||
|
||||
Systematic root cause analysis:
|
||||
- 5 Whys, Fishbone diagrams
|
||||
- Solution generation
|
||||
- Impact assessment
|
||||
|
||||
### Innovation Strategy
|
||||
|
||||
Business model disruption:
|
||||
- Blue Ocean Strategy
|
||||
- Jobs-to-be-Done
|
||||
- Disruptive innovation patterns
|
||||
|
||||
### Storytelling
|
||||
|
||||
25 narrative frameworks:
|
||||
- Hero's Journey
|
||||
- Story circles
|
||||
- Compelling pitch structures
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Direct Workflow
|
||||
|
||||
```bash
|
||||
workflow brainstorming
|
||||
|
||||
workflow design-thinking --data /path/to/context.md
|
||||
```
|
||||
|
||||
### Agent-Facilitated
|
||||
|
||||
```bash
|
||||
agent cis/brainstorming-coach
|
||||
|
||||
> *brainstorm
|
||||
```
|
||||
|
||||
## Key Differentiators
|
||||
|
||||
- **Facilitation Over Generation** - Guides discovery through questions
|
||||
- **Energy-Aware Sessions** - Adapts to engagement levels
|
||||
- **Context Integration** - Domain-specific guidance support
|
||||
- **Persona-Driven** - Unique communication styles
|
||||
- **Rich Method Libraries** - 150+ proven techniques
|
||||
|
||||
## Integration Points
|
||||
|
||||
CIS workflows integrate with:
|
||||
|
||||
- **BMM** - Powers project brainstorming
|
||||
- **BMB** - Creative module design
|
||||
- **Custom Modules** - Shared creative resource
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Set clear objectives** before starting sessions
|
||||
2. **Provide context documents** for domain relevance
|
||||
3. **Trust the process** - Let facilitation guide you
|
||||
4. **Take breaks** when energy flags
|
||||
5. **Document insights** as they emerge
|
||||
|
||||
:::tip[Learn More]
|
||||
See [Facilitation Over Generation](/docs/explanation/philosophy/facilitation-over-generation.md) for the core philosophy behind CIS.
|
||||
:::
|
||||
|
|
@ -9,21 +9,45 @@ Quick answers to common questions about tools, IDEs, and advanced topics in the
|
|||
|
||||
**Tools and Technical**
|
||||
|
||||
- [Why are my Mermaid diagrams not rendering?](#why-are-my-mermaid-diagrams-not-rendering)
|
||||
- [Can I use BMM with GitHub Copilot / Cursor / other AI tools?](#can-i-use-bmm-with-github-copilot--cursor--other-ai-tools)
|
||||
- [What IDEs/tools support BMM?](#what-idestools-support-bmm)
|
||||
- [Can I customize agents?](#can-i-customize-agents)
|
||||
- [What happens to my planning docs after implementation?](#what-happens-to-my-planning-docs-after-implementation)
|
||||
- [Can I use BMM for non-software projects?](#can-i-use-bmm-for-non-software-projects)
|
||||
- [Questions](#questions)
|
||||
- [Tools and Technical](#tools-and-technical)
|
||||
- [Why are my Mermaid diagrams not rendering?](#why-are-my-mermaid-diagrams-not-rendering)
|
||||
- [Can I use BMM with GitHub Copilot / Cursor / other AI tools?](#can-i-use-bmm-with-github-copilot--cursor--other-ai-tools)
|
||||
- [What IDEs/tools support BMM?](#what-idestools-support-bmm)
|
||||
- [Can I customize agents?](#can-i-customize-agents)
|
||||
- [What happens to my planning docs after implementation?](#what-happens-to-my-planning-docs-after-implementation)
|
||||
- [Can I use BMM for non-software projects?](#can-i-use-bmm-for-non-software-projects)
|
||||
- [Advanced](#advanced)
|
||||
- [What if my project grows from Level 1 to Level 3?](#what-if-my-project-grows-from-level-1-to-level-3)
|
||||
- [Can I mix greenfield and brownfield approaches?](#can-i-mix-greenfield-and-brownfield-approaches)
|
||||
- [How do I handle urgent hotfixes during a sprint?](#how-do-i-handle-urgent-hotfixes-during-a-sprint)
|
||||
- [What if I disagree with the workflow's recommendations?](#what-if-i-disagree-with-the-workflows-recommendations)
|
||||
- [Can multiple developers work on the same BMM project?](#can-multiple-developers-work-on-the-same-bmm-project)
|
||||
- [What is party mode and when should I use it?](#what-is-party-mode-and-when-should-i-use-it)
|
||||
- [Getting Help](#getting-help)
|
||||
- [Where do I get help if my question isn't answered here?](#where-do-i-get-help-if-my-question-isnt-answered-here)
|
||||
- [How do I report a bug or request a feature?](#how-do-i-report-a-bug-or-request-a-feature)
|
||||
|
||||
**Advanced**
|
||||
|
||||
- [What if my project grows from Level 1 to Level 3?](#what-if-my-project-grows-from-level-1-to-level-3)
|
||||
- [Can I mix greenfield and brownfield approaches?](#can-i-mix-greenfield-and-brownfield-approaches)
|
||||
- [How do I handle urgent hotfixes during a sprint?](#how-do-i-handle-urgent-hotfixes-during-a-sprint)
|
||||
- [What if I disagree with the workflow's recommendations?](#what-if-i-disagree-with-the-workflows-recommendations)
|
||||
- [Can multiple developers work on the same BMM project?](#can-multiple-developers-work-on-the-same-bmm-project)
|
||||
- [What is party mode and when should I use it?](#what-is-party-mode-and-when-should-i-use-it)
|
||||
- [Questions](#questions)
|
||||
- [Tools and Technical](#tools-and-technical)
|
||||
- [Why are my Mermaid diagrams not rendering?](#why-are-my-mermaid-diagrams-not-rendering)
|
||||
- [Can I use BMM with GitHub Copilot / Cursor / other AI tools?](#can-i-use-bmm-with-github-copilot--cursor--other-ai-tools)
|
||||
- [What IDEs/tools support BMM?](#what-idestools-support-bmm)
|
||||
- [Can I customize agents?](#can-i-customize-agents)
|
||||
- [What happens to my planning docs after implementation?](#what-happens-to-my-planning-docs-after-implementation)
|
||||
- [Can I use BMM for non-software projects?](#can-i-use-bmm-for-non-software-projects)
|
||||
- [Advanced](#advanced)
|
||||
- [What if my project grows from Level 1 to Level 3?](#what-if-my-project-grows-from-level-1-to-level-3)
|
||||
- [Can I mix greenfield and brownfield approaches?](#can-i-mix-greenfield-and-brownfield-approaches)
|
||||
- [How do I handle urgent hotfixes during a sprint?](#how-do-i-handle-urgent-hotfixes-during-a-sprint)
|
||||
- [What if I disagree with the workflow's recommendations?](#what-if-i-disagree-with-the-workflows-recommendations)
|
||||
- [Can multiple developers work on the same BMM project?](#can-multiple-developers-work-on-the-same-bmm-project)
|
||||
- [What is party mode and when should I use it?](#what-is-party-mode-and-when-should-i-use-it)
|
||||
- [Getting Help](#getting-help)
|
||||
- [Where do I get help if my question isn't answered here?](#where-do-i-get-help-if-my-question-isnt-answered-here)
|
||||
- [How do I report a bug or request a feature?](#how-do-i-report-a-bug-or-request-a-feature)
|
||||
|
||||
**Getting Help**
|
||||
|
||||
|
|
@ -199,11 +223,11 @@ Yes! But the paradigm is fundamentally different from traditional agile teams.
|
|||
|
||||
### What is party mode and when should I use it?
|
||||
|
||||
Party mode is a unique multi-agent collaboration feature where ALL your installed agents (19+ from BMM, CIS, BMB, custom modules) discuss your challenges together in real-time.
|
||||
Party mode is a unique multi-agent collaboration feature where ALL your installed modules agents discuss your challenges together in real-time or have some fun with any topic you have in mind.
|
||||
|
||||
**How it works:**
|
||||
|
||||
1. Run `/bmad:core:workflows:party-mode` (or `*party-mode` from any agent)
|
||||
1. Run `/bmad:core:workflows:party-mode` (or `PM or fuzzy match on party-mode` from any agent)
|
||||
2. Introduce your topic
|
||||
3. BMad Master selects 2-3 most relevant agents per message
|
||||
4. Agents cross-talk, debate, and build on each other's ideas
|
||||
|
|
|
|||
|
|
@ -23,11 +23,16 @@ BMad does not mandate TEA. There are five valid ways to use it (or skip it). Pic
|
|||
1. **No TEA**
|
||||
- Skip all TEA workflows. Use your existing team testing approach.
|
||||
|
||||
2. **TEA-only (Standalone)**
|
||||
2. **TEA Solo (Standalone)**
|
||||
- Use TEA on a non-BMad project. Bring your own requirements, acceptance criteria, and environments.
|
||||
- Typical sequence: `*test-design` (system or epic) -> `*atdd` and/or `*automate` -> optional `*test-review` -> `*trace` for coverage and gate decisions.
|
||||
- Run `*framework` or `*ci` only if you want TEA to scaffold the harness or pipeline; they work best after you decide the stack/architecture.
|
||||
|
||||
**TEA Lite (Beginner Approach):**
|
||||
- Simplest way to use TEA - just use `*automate` to test existing features.
|
||||
- Perfect for learning TEA fundamentals in 30 minutes.
|
||||
- See [TEA Lite Quickstart Tutorial](/docs/tutorials/getting-started/tea-lite-quickstart.md).
|
||||
|
||||
3. **Integrated: Greenfield - BMad Method (Simple/Standard Work)**
|
||||
- Phase 3: system-level `*test-design`, then `*framework` and `*ci`.
|
||||
- Phase 4: per-epic `*test-design`, optional `*atdd`, then `*automate` and optional `*test-review`.
|
||||
|
|
@ -50,16 +55,16 @@ If you are unsure, default to the integrated path for your track and adjust late
|
|||
|
||||
## TEA Command Catalog
|
||||
|
||||
| Command | Primary Outputs | Notes | With Playwright MCP Enhancements |
|
||||
| -------------- | --------------------------------------------------------------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| `*framework` | Playwright/Cypress scaffold, `.env.example`, `.nvmrc`, sample specs | Use when no production-ready harness exists | - |
|
||||
| `*ci` | CI workflow, selective test scripts, secrets checklist | Platform-aware (GitHub Actions default) | - |
|
||||
| `*test-design` | Combined risk assessment, mitigation plan, and coverage strategy | Risk scoring + optional exploratory mode | **+ Exploratory**: Interactive UI discovery with browser automation (uncover actual functionality) |
|
||||
| `*atdd` | Failing acceptance tests + implementation checklist | TDD red phase + optional recording mode | **+ Recording**: AI generation verified with live browser (accurate selectors from real DOM) |
|
||||
| `*automate` | Prioritized specs, fixtures, README/script updates, DoD summary | Optional healing/recording, avoid duplicate coverage | **+ Healing**: Pattern fixes enhanced with visual debugging + **+ Recording**: AI verified with live browser |
|
||||
| `*test-review` | Test quality review report with 0-100 score, violations, fixes | Reviews tests against knowledge base patterns | - |
|
||||
| `*nfr-assess` | NFR assessment report with actions | Focus on security/performance/reliability | - |
|
||||
| `*trace` | Phase 1: Coverage matrix, recommendations. Phase 2: Gate decision (PASS/CONCERNS/FAIL/WAIVED) | Two-phase workflow: traceability + gate decision | - |
|
||||
| Command | Primary Outputs | Notes | With Playwright MCP Enhancements |
|
||||
| -------------- | --------------------------------------------------------------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `*framework` | Playwright/Cypress scaffold, `.env.example`, `.nvmrc`, sample specs | Use when no production-ready harness exists | - |
|
||||
| `*ci` | CI workflow, selective test scripts, secrets checklist | Platform-aware (GitHub Actions default) | - |
|
||||
| `*test-design` | Combined risk assessment, mitigation plan, and coverage strategy | Risk scoring + optional exploratory mode | **+ Exploratory**: Interactive UI discovery with browser automation (uncover actual functionality) |
|
||||
| `*atdd` | Failing acceptance tests + implementation checklist | TDD red phase + optional recording mode | **+ Recording**: UI selectors verified with live browser; API tests benefit from trace analysis |
|
||||
| `*automate` | Prioritized specs, fixtures, README/script updates, DoD summary | Optional healing/recording, avoid duplicate coverage | **+ Healing**: Visual debugging + trace analysis for test fixes; **+ Recording**: Verified selectors (UI) + network inspection (API) |
|
||||
| `*test-review` | Test quality review report with 0-100 score, violations, fixes | Reviews tests against knowledge base patterns | - |
|
||||
| `*nfr-assess` | NFR assessment report with actions | Focus on security/performance/reliability | - |
|
||||
| `*trace` | Phase 1: Coverage matrix, recommendations. Phase 2: Gate decision (PASS/CONCERNS/FAIL/WAIVED) | Two-phase workflow: traceability + gate decision | - |
|
||||
|
||||
## TEA Workflow Lifecycle
|
||||
|
||||
|
|
@ -168,12 +173,12 @@ TEA spans multiple phases (Phase 3, Phase 4, and the release gate). Most BMM age
|
|||
|
||||
### TEA's 8 Workflows Across Phases
|
||||
|
||||
| Phase | TEA Workflows | Frequency | Purpose |
|
||||
| ----------- | --------------------------------------------------------- | ---------------- | ---------------------------------------------- |
|
||||
| **Phase 2** | (none) | - | Planning phase - PM defines requirements |
|
||||
| Phase | TEA Workflows | Frequency | Purpose |
|
||||
| ----------- | --------------------------------------------------------- | ---------------- | ------------------------------------------------------- |
|
||||
| **Phase 2** | (none) | - | Planning phase - PM defines requirements |
|
||||
| **Phase 3** | \*test-design (system-level), \*framework, \*ci | Once per project | System testability review and test infrastructure setup |
|
||||
| **Phase 4** | \*test-design, \*atdd, \*automate, \*test-review, \*trace | Per epic/story | Test planning per epic, then per-story testing |
|
||||
| **Release** | \*nfr-assess, \*trace (Phase 2: gate) | Per epic/release | Go/no-go decision |
|
||||
| **Phase 4** | \*test-design, \*atdd, \*automate, \*test-review, \*trace | Per epic/story | Test planning per epic, then per-story testing |
|
||||
| **Release** | \*nfr-assess, \*trace (Phase 2: gate) | Per epic/release | Go/no-go decision |
|
||||
|
||||
**Note**: `*trace` is a two-phase workflow: Phase 1 (traceability) + Phase 2 (gate decision). This reduces cognitive load while maintaining natural workflow.
|
||||
|
||||
|
|
@ -279,6 +284,31 @@ These cheat sheets map TEA workflows to the **BMad Method and Enterprise tracks*
|
|||
**Related how-to guides:**
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md)
|
||||
- [How to Set Up a Test Framework](/docs/how-to/workflows/setup-test-framework.md)
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md)
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md)
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md)
|
||||
- [How to Set Up CI Pipeline](/docs/how-to/workflows/setup-ci.md)
|
||||
- [How to Run NFR Assessment](/docs/how-to/workflows/run-nfr-assess.md)
|
||||
- [How to Run Trace](/docs/how-to/workflows/run-trace.md)
|
||||
|
||||
## Deep Dive Concepts
|
||||
|
||||
Want to understand TEA principles and patterns in depth?
|
||||
|
||||
**Core Principles:**
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Probability × impact scoring, P0-P3 priorities
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Definition of Done, determinism, isolation
|
||||
- [Knowledge Base System](/docs/explanation/tea/knowledge-base-system.md) - Context engineering with tea-index.csv
|
||||
|
||||
**Technical Patterns:**
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Pure function → fixture → composition
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Eliminating flakiness with intercept-before-navigate
|
||||
|
||||
**Engagement & Strategy:**
|
||||
- [Engagement Models](/docs/explanation/tea/engagement-models.md) - TEA Lite, TEA Solo, TEA Integrated (5 models explained)
|
||||
|
||||
**Philosophy:**
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - **Start here to understand WHY TEA exists** - The problem with AI-generated tests and TEA's three-part solution
|
||||
|
||||
## Optional Integrations
|
||||
|
||||
|
|
@ -322,3 +352,59 @@ Live browser verification for test design and automation.
|
|||
- Enhances healing with `browser_snapshot`, console, network, and locator tools.
|
||||
|
||||
**To disable**: set `tea_use_mcp_enhancements: false` in `_bmad/bmm/config.yaml` or remove MCPs from IDE config.
|
||||
|
||||
---
|
||||
|
||||
## Complete TEA Documentation Navigation
|
||||
|
||||
### Start Here
|
||||
|
||||
**New to TEA? Start with the tutorial:**
|
||||
- [TEA Lite Quickstart Tutorial](/docs/tutorials/getting-started/tea-lite-quickstart.md) - 30-minute beginner guide using TodoMVC
|
||||
|
||||
### Workflow Guides (Task-Oriented)
|
||||
|
||||
**All 8 TEA workflows with step-by-step instructions:**
|
||||
1. [How to Set Up a Test Framework with TEA](/docs/how-to/workflows/setup-test-framework.md) - Scaffold Playwright or Cypress
|
||||
2. [How to Set Up CI Pipeline with TEA](/docs/how-to/workflows/setup-ci.md) - Configure CI/CD with selective testing
|
||||
3. [How to Run Test Design with TEA](/docs/how-to/workflows/run-test-design.md) - Risk-based test planning (system or epic)
|
||||
4. [How to Run ATDD with TEA](/docs/how-to/workflows/run-atdd.md) - Generate failing tests before implementation
|
||||
5. [How to Run Automate with TEA](/docs/how-to/workflows/run-automate.md) - Expand test coverage after implementation
|
||||
6. [How to Run Test Review with TEA](/docs/how-to/workflows/run-test-review.md) - Audit test quality (0-100 scoring)
|
||||
7. [How to Run NFR Assessment with TEA](/docs/how-to/workflows/run-nfr-assess.md) - Validate non-functional requirements
|
||||
8. [How to Run Trace with TEA](/docs/how-to/workflows/run-trace.md) - Coverage traceability + gate decisions
|
||||
|
||||
### Customization & Integration
|
||||
|
||||
**Optional enhancements to TEA workflows:**
|
||||
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) - Production-ready fixtures and 9 utilities
|
||||
- [Enable TEA MCP Enhancements](/docs/how-to/customization/enable-tea-mcp-enhancements.md) - Live browser verification, visual debugging
|
||||
|
||||
### Use-Case Guides
|
||||
|
||||
**Specialized guidance for specific contexts:**
|
||||
- [Using TEA with Existing Tests (Brownfield)](/docs/how-to/brownfield/use-tea-with-existing-tests.md) - Incremental improvement, regression hotspots, baseline coverage
|
||||
- [Running TEA for Enterprise](/docs/how-to/brownfield/use-tea-for-enterprise.md) - Compliance, NFR assessment, audit trails, SOC 2/HIPAA
|
||||
|
||||
### Concept Deep Dives (Understanding-Oriented)
|
||||
|
||||
**Understand the principles and patterns:**
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Probability × impact scoring, P0-P3 priorities, mitigation strategies
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Definition of Done, determinism, isolation, explicit assertions
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Pure function → fixture → composition pattern
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Intercept-before-navigate, eliminating flakiness
|
||||
- [Knowledge Base System](/docs/explanation/tea/knowledge-base-system.md) - Context engineering with tea-index.csv, 33 fragments
|
||||
- [Engagement Models](/docs/explanation/tea/engagement-models.md) - TEA Lite, TEA Solo, TEA Integrated (5 models explained)
|
||||
|
||||
### Philosophy & Design
|
||||
|
||||
**Why TEA exists and how it works:**
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - **Start here to understand WHY** - The problem with AI-generated tests and TEA's three-part solution
|
||||
|
||||
### Reference (Quick Lookup)
|
||||
|
||||
**Factual information for quick reference:**
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - All 8 workflows: inputs, outputs, phases, frequency
|
||||
- [TEA Configuration Reference](/docs/reference/tea/configuration.md) - Config options, file locations, setup examples
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - 33 fragments categorized and explained
|
||||
- [Glossary - TEA Section](/docs/reference/glossary/index.md#test-architect-tea-concepts) - 20 TEA-specific terms defined
|
||||
|
|
|
|||
|
|
@ -1,387 +0,0 @@
|
|||
---
|
||||
title: "BMGD Agents Guide"
|
||||
---
|
||||
|
||||
Complete reference for BMGD's six specialized game development agents.
|
||||
|
||||
## Agent Overview
|
||||
|
||||
BMGD provides six agents, each with distinct expertise:
|
||||
|
||||
| Agent | Name | Role | Phase Focus |
|
||||
|-------|------|------|-------------|
|
||||
| **Game Designer** | Samus Shepard | Lead Game Designer + Creative Vision Architect | Phases 1-2 |
|
||||
| **Game Architect** | Cloud Dragonborn | Principal Game Systems Architect + Technical Director | Phase 3 |
|
||||
| **Game Developer** | Link Freeman | Senior Game Developer + Technical Implementation Specialist | Phase 4 |
|
||||
| **Game Scrum Master** | Max | Game Development Scrum Master + Sprint Orchestrator | Phase 4 |
|
||||
| **Game QA** | GLaDOS | Game QA Architect + Test Automation Specialist | All Phases |
|
||||
| **Game Solo Dev** | Indie | Elite Indie Game Developer + Quick Flow Specialist | All Phases |
|
||||
|
||||
## Game Designer (Samus Shepard)
|
||||
|
||||
### Role
|
||||
|
||||
Lead Game Designer + Creative Vision Architect
|
||||
|
||||
### Identity
|
||||
|
||||
Veteran designer with 15+ years crafting AAA and indie hits. Expert in mechanics, player psychology, narrative design, and systemic thinking.
|
||||
|
||||
### Communication Style
|
||||
|
||||
Talks like an excited streamer - enthusiastic, asks about player motivations, celebrates breakthroughs with "Let's GOOO!"
|
||||
|
||||
### Core Principles
|
||||
|
||||
- Design what players want to FEEL, not what they say they want
|
||||
- Prototype fast - one hour of playtesting beats ten hours of discussion
|
||||
- Every mechanic must serve the core fantasy
|
||||
|
||||
### When to Use
|
||||
|
||||
- Brainstorming game ideas
|
||||
- Creating Game Briefs
|
||||
- Designing GDDs
|
||||
- Developing narrative design
|
||||
|
||||
### Available Commands
|
||||
|
||||
| Command | Description |
|
||||
| ---------------------- | -------------------------------- |
|
||||
| `workflow-status` | Check project status |
|
||||
| `brainstorm-game` | Guided game ideation |
|
||||
| `create-game-brief` | Create Game Brief |
|
||||
| `create-gdd` | Create Game Design Document |
|
||||
| `narrative` | Create Narrative Design Document |
|
||||
| `quick-prototype` | Rapid prototyping (IDE only) |
|
||||
| `party-mode` | Multi-agent collaboration |
|
||||
| `advanced-elicitation` | Deep exploration (web only) |
|
||||
|
||||
## Game Architect (Cloud Dragonborn)
|
||||
|
||||
### Role
|
||||
|
||||
Principal Game Systems Architect + Technical Director
|
||||
|
||||
### Identity
|
||||
|
||||
Master architect with 20+ years shipping 30+ titles. Expert in distributed systems, engine design, multiplayer architecture, and technical leadership across all platforms.
|
||||
|
||||
### Communication Style
|
||||
|
||||
Speaks like a wise sage from an RPG - calm, measured, uses architectural metaphors about building foundations and load-bearing walls.
|
||||
|
||||
### Core Principles
|
||||
|
||||
- Architecture is about delaying decisions until you have enough data
|
||||
- Build for tomorrow without over-engineering today
|
||||
- Hours of planning save weeks of refactoring hell
|
||||
- Every system must handle the hot path at 60fps
|
||||
|
||||
### When to Use
|
||||
|
||||
- Planning technical architecture
|
||||
- Making engine/framework decisions
|
||||
- Designing game systems
|
||||
- Course correction during development
|
||||
|
||||
### Available Commands
|
||||
|
||||
| Command | Description |
|
||||
| ---------------------- | ------------------------------------- |
|
||||
| `workflow-status` | Check project status |
|
||||
| `create-architecture` | Create Game Architecture |
|
||||
| `correct-course` | Course correction analysis (IDE only) |
|
||||
| `party-mode` | Multi-agent collaboration |
|
||||
| `advanced-elicitation` | Deep exploration (web only) |
|
||||
|
||||
## Game Developer (Link Freeman)
|
||||
|
||||
### Role
|
||||
|
||||
Senior Game Developer + Technical Implementation Specialist
|
||||
|
||||
### Identity
|
||||
|
||||
Battle-hardened dev with expertise in Unity, Unreal, and custom engines. Ten years shipping across mobile, console, and PC. Writes clean, performant code.
|
||||
|
||||
### Communication Style
|
||||
|
||||
Speaks like a speedrunner - direct, milestone-focused, always optimizing for the fastest path to ship.
|
||||
|
||||
### Core Principles
|
||||
|
||||
- 60fps is non-negotiable
|
||||
- Write code designers can iterate without fear
|
||||
- Ship early, ship often, iterate on player feedback
|
||||
- Red-green-refactor: tests first, implementation second
|
||||
|
||||
### When to Use
|
||||
|
||||
- Implementing stories
|
||||
- Code reviews
|
||||
- Performance optimization
|
||||
- Completing story work
|
||||
|
||||
### Available Commands
|
||||
|
||||
| Command | Description |
|
||||
| ---------------------- | ------------------------------- |
|
||||
| `workflow-status` | Check sprint progress |
|
||||
| `dev-story` | Implement story tasks |
|
||||
| `code-review` | Perform code review |
|
||||
| `quick-dev` | Flexible development (IDE only) |
|
||||
| `quick-prototype` | Rapid prototyping (IDE only) |
|
||||
| `party-mode` | Multi-agent collaboration |
|
||||
| `advanced-elicitation` | Deep exploration (web only) |
|
||||
|
||||
## Game Scrum Master (Max)
|
||||
|
||||
### Role
|
||||
|
||||
Game Development Scrum Master + Sprint Orchestrator
|
||||
|
||||
### Identity
|
||||
|
||||
Certified Scrum Master specializing in game dev workflows. Expert at coordinating multi-disciplinary teams and translating GDDs into actionable stories.
|
||||
|
||||
### Communication Style
|
||||
|
||||
Talks in game terminology - milestones are save points, handoffs are level transitions, blockers are boss fights.
|
||||
|
||||
### Core Principles
|
||||
|
||||
- Every sprint delivers playable increments
|
||||
- Clean separation between design and implementation
|
||||
- Keep the team moving through each phase
|
||||
- Stories are single source of truth for implementation
|
||||
|
||||
### When to Use
|
||||
|
||||
- Sprint planning and management
|
||||
- Creating epic tech specs
|
||||
- Writing story drafts
|
||||
- Assembling story context
|
||||
- Running retrospectives
|
||||
- Handling course corrections
|
||||
|
||||
### Available Commands
|
||||
|
||||
| Command | Description |
|
||||
| ----------------------- | ------------------------------------------- |
|
||||
| `workflow-status` | Check project status |
|
||||
| `sprint-planning` | Generate/update sprint status |
|
||||
| `sprint-status` | View sprint progress, get next action |
|
||||
| `create-story` | Create story (marks ready-for-dev directly) |
|
||||
| `validate-create-story` | Validate story draft |
|
||||
| `epic-retrospective` | Facilitate retrospective |
|
||||
| `correct-course` | Navigate significant changes |
|
||||
| `party-mode` | Multi-agent collaboration |
|
||||
| `advanced-elicitation` | Deep exploration (web only) |
|
||||
|
||||
## Game QA (GLaDOS)
|
||||
|
||||
### Role
|
||||
|
||||
Game QA Architect + Test Automation Specialist
|
||||
|
||||
### Identity
|
||||
|
||||
Senior QA architect with 12+ years in game testing across Unity, Unreal, and Godot. Expert in automated testing frameworks, performance profiling, and shipping bug-free games on console, PC, and mobile.
|
||||
|
||||
### Communication Style
|
||||
|
||||
Speaks like a quality guardian - methodical, data-driven, but understands that "feel" matters in games. Uses metrics to back intuition. "Trust, but verify with tests."
|
||||
|
||||
### Core Principles
|
||||
|
||||
- Test what matters: gameplay feel, performance, progression
|
||||
- Automated tests catch regressions, humans catch fun problems
|
||||
- Every shipped bug is a process failure, not a people failure
|
||||
- Flaky tests are worse than no tests - they erode trust
|
||||
- Profile before optimize, test before ship
|
||||
|
||||
### When to Use
|
||||
|
||||
- Setting up test frameworks
|
||||
- Designing test strategies
|
||||
- Creating automated tests
|
||||
- Planning playtesting sessions
|
||||
- Performance testing
|
||||
- Reviewing test coverage
|
||||
|
||||
### Available Commands
|
||||
|
||||
| Command | Description |
|
||||
| ---------------------- | --------------------------------------------------- |
|
||||
| `workflow-status` | Check project status |
|
||||
| `test-framework` | Initialize game test framework (Unity/Unreal/Godot) |
|
||||
| `test-design` | Create comprehensive game test scenarios |
|
||||
| `automate` | Generate automated game tests |
|
||||
| `playtest-plan` | Create structured playtesting plan |
|
||||
| `performance-test` | Design performance testing strategy |
|
||||
| `test-review` | Review test quality and coverage |
|
||||
| `party-mode` | Multi-agent collaboration |
|
||||
| `advanced-elicitation` | Deep exploration (web only) |
|
||||
|
||||
### Knowledge Base
|
||||
|
||||
GLaDOS has access to a comprehensive game testing knowledge base (`gametest/qa-index.csv`) including:
|
||||
|
||||
**Engine-Specific Testing:**
|
||||
|
||||
- Unity Test Framework (Edit Mode, Play Mode)
|
||||
- Unreal Automation and Gauntlet
|
||||
- Godot GUT (Godot Unit Test)
|
||||
|
||||
**Game-Specific Testing:**
|
||||
|
||||
- Playtesting fundamentals
|
||||
- Balance testing
|
||||
- Save system testing
|
||||
- Multiplayer/network testing
|
||||
- Input testing
|
||||
- Platform certification (TRC/XR)
|
||||
- Localization testing
|
||||
|
||||
**General QA:**
|
||||
|
||||
- QA automation strategies
|
||||
- Performance testing
|
||||
- Regression testing
|
||||
- Smoke testing
|
||||
- Test prioritization (P0-P3)
|
||||
|
||||
## Game Solo Dev (Indie)
|
||||
|
||||
### Role
|
||||
|
||||
Elite Indie Game Developer + Quick Flow Specialist
|
||||
|
||||
### Identity
|
||||
|
||||
Battle-hardened solo game developer who ships complete games from concept to launch. Expert in Unity, Unreal, and Godot, having shipped titles across mobile, PC, and console. Lives and breathes the Quick Flow workflow - prototyping fast, iterating faster, and shipping before the hype dies.
|
||||
|
||||
### Communication Style
|
||||
|
||||
Direct, confident, and gameplay-focused. Uses dev slang, thinks in game feel and player experience. Every response moves the game closer to ship. "Does it feel good? Ship it."
|
||||
|
||||
### Core Principles
|
||||
|
||||
- Prototype fast, fail fast, iterate faster
|
||||
- A playable build beats a perfect design doc
|
||||
- 60fps is non-negotiable - performance is a feature
|
||||
- The core loop must be fun before anything else matters
|
||||
- Ship early, playtest often
|
||||
|
||||
### When to Use
|
||||
|
||||
- Solo game development
|
||||
- Rapid prototyping
|
||||
- Quick iteration without full team workflow
|
||||
- Indie projects with tight timelines
|
||||
- When you want to handle everything yourself
|
||||
|
||||
### Available Commands
|
||||
|
||||
| Command | Description |
|
||||
| ------------------ | ------------------------------------------------------ |
|
||||
| `quick-prototype` | Rapid prototype to test if a mechanic is fun |
|
||||
| `quick-dev` | Implement features end-to-end with game considerations |
|
||||
| `quick-spec` | Create implementation-ready technical spec |
|
||||
| `code-review` | Review code quality |
|
||||
| `test-framework` | Set up automated testing |
|
||||
| `party-mode` | Bring in specialists when needed |
|
||||
|
||||
### Quick Flow vs Full BMGD
|
||||
|
||||
Use **Game Solo Dev** when:
|
||||
|
||||
- You're working alone or in a tiny team
|
||||
- Speed matters more than process
|
||||
- You want to skip the full planning phases
|
||||
- You're prototyping or doing game jams
|
||||
|
||||
Use **Full BMGD workflow** when:
|
||||
|
||||
- You have a larger team
|
||||
- The project needs formal documentation
|
||||
- You're working with stakeholders/publishers
|
||||
- Long-term maintainability is critical
|
||||
|
||||
## Agent Selection Guide
|
||||
|
||||
### By Phase
|
||||
|
||||
| Phase | Primary Agent | Secondary Agent |
|
||||
| ------------------------------ | ----------------- | ----------------- |
|
||||
| 1: Preproduction | Game Designer | - |
|
||||
| 2: Design | Game Designer | - |
|
||||
| 3: Technical | Game Architect | Game QA |
|
||||
| 4: Production (Planning) | Game Scrum Master | Game Architect |
|
||||
| 4: Production (Implementation) | Game Developer | Game Scrum Master |
|
||||
| Testing (Any Phase) | Game QA | Game Developer |
|
||||
|
||||
### By Task
|
||||
|
||||
| Task | Best Agent |
|
||||
| -------------------------------- | ----------------- |
|
||||
| "I have a game idea" | Game Designer |
|
||||
| "Help me design my game" | Game Designer |
|
||||
| "How should I build this?" | Game Architect |
|
||||
| "What's the technical approach?" | Game Architect |
|
||||
| "Plan our sprints" | Game Scrum Master |
|
||||
| "Create implementation stories" | Game Scrum Master |
|
||||
| "Build this feature" | Game Developer |
|
||||
| "Review this code" | Game Developer |
|
||||
| "Set up testing framework" | Game QA |
|
||||
| "Create test plan" | Game QA |
|
||||
| "Test performance" | Game QA |
|
||||
| "Plan a playtest" | Game QA |
|
||||
| "I'm working solo" | Game Solo Dev |
|
||||
| "Quick prototype this idea" | Game Solo Dev |
|
||||
| "Ship this feature fast" | Game Solo Dev |
|
||||
|
||||
## Multi-Agent Collaboration
|
||||
|
||||
### Party Mode
|
||||
|
||||
All agents have access to `party-mode`, which brings multiple agents together for complex decisions. Use this when:
|
||||
|
||||
- A decision spans multiple domains (design + technical)
|
||||
- You want diverse perspectives
|
||||
- You're stuck and need fresh ideas
|
||||
|
||||
### Handoffs
|
||||
|
||||
Agents naturally hand off to each other:
|
||||
|
||||
```
|
||||
Game Designer → Game Architect → Game Scrum Master → Game Developer
|
||||
↓ ↓ ↓ ↓
|
||||
GDD Architecture Sprint/Stories Implementation
|
||||
↓ ↓
|
||||
Game QA ←──────────────────────────── Game QA
|
||||
↓ ↓
|
||||
Test Strategy Automated Tests
|
||||
```
|
||||
|
||||
Game QA integrates at multiple points:
|
||||
|
||||
- After Architecture: Define test strategy
|
||||
- During Implementation: Create automated tests
|
||||
- Before Release: Performance and certification testing
|
||||
|
||||
## Project Context
|
||||
|
||||
All agents share the principle:
|
||||
|
||||
> "Find if this exists, if it does, always treat it as the bible I plan and execute against: `**/project-context.md`"
|
||||
|
||||
The `project-context.md` file (if present) serves as the authoritative source for project decisions and constraints.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **[Quick Start Guide](/docs/tutorials/getting-started/quick-start-bmgd.md)** - Get started with BMGD
|
||||
- **[Workflows Guide](/docs/reference/workflows/index.md)** - Detailed workflow reference
|
||||
- **[Game Types Guide](/docs/explanation/game-dev/game-types.md)** - Game type templates
|
||||
|
|
@ -1,125 +0,0 @@
|
|||
---
|
||||
title: "BMGD vs BMM"
|
||||
description: Understanding the differences between BMGD and BMM
|
||||
---
|
||||
|
||||
BMGD (BMad Game Development) extends BMM (BMad Method) with game-specific capabilities. This page explains the key differences.
|
||||
|
||||
## Quick Comparison
|
||||
|
||||
| Aspect | BMM | BMGD |
|
||||
| -------------- | ------------------------------------- | ------------------------------------------------------------------------ |
|
||||
| **Focus** | General software | Game development |
|
||||
| **Agents** | PM, Architect, Dev, SM, TEA, Solo Dev | Game Designer, Game Dev, Game Architect, Game SM, Game QA, Game Solo Dev |
|
||||
| **Planning** | PRD, Tech Spec | Game Brief, GDD |
|
||||
| **Types** | N/A | 24 game type templates |
|
||||
| **Narrative** | N/A | Full narrative workflow |
|
||||
| **Testing** | Web-focused | Engine-specific (Unity, Unreal, Godot) |
|
||||
| **Production** | BMM workflows | BMM workflows with game overrides |
|
||||
|
||||
## Agent Differences
|
||||
|
||||
### BMM Agents
|
||||
- PM (Product Manager)
|
||||
- Architect
|
||||
- DEV (Developer)
|
||||
- SM (Scrum Master)
|
||||
- TEA (Test Architect)
|
||||
- Quick Flow Solo Dev
|
||||
|
||||
### BMGD Agents
|
||||
- Game Designer
|
||||
- Game Developer
|
||||
- Game Architect
|
||||
- Game Scrum Master
|
||||
- Game QA
|
||||
- Game Solo Dev
|
||||
|
||||
BMGD agents understand game-specific concepts like:
|
||||
- Game mechanics and balance
|
||||
- Player psychology
|
||||
- Engine-specific patterns
|
||||
- Playtesting and QA
|
||||
|
||||
## Planning Documents
|
||||
|
||||
### BMM Planning
|
||||
- **Product Brief** → **PRD** → **Architecture**
|
||||
- Focus: Software requirements, user stories, system design
|
||||
|
||||
### BMGD Planning
|
||||
- **Game Brief** → **GDD** → **Architecture**
|
||||
- Focus: Game vision, mechanics, narrative, player experience
|
||||
|
||||
The GDD (Game Design Document) includes:
|
||||
- Core gameplay loop
|
||||
- Mechanics and systems
|
||||
- Progression and balance
|
||||
- Art and audio direction
|
||||
- Genre-specific sections
|
||||
|
||||
## Game Type Templates
|
||||
|
||||
BMGD includes 24 game type templates that auto-configure GDD sections:
|
||||
|
||||
- Action, Adventure, Puzzle
|
||||
- RPG, Strategy, Simulation
|
||||
- Sports, Racing, Fighting
|
||||
- Horror, Platformer, Shooter
|
||||
- And more...
|
||||
|
||||
Each template provides:
|
||||
- Genre-specific GDD sections
|
||||
- Relevant mechanics patterns
|
||||
- Testing considerations
|
||||
- Common pitfalls to avoid
|
||||
|
||||
## Narrative Support
|
||||
|
||||
BMGD includes full narrative workflow for story-driven games:
|
||||
|
||||
- **Narrative Design** workflow
|
||||
- Story structure templates
|
||||
- Character development
|
||||
- World-building guidelines
|
||||
- Dialogue systems
|
||||
|
||||
BMM has no equivalent for narrative design.
|
||||
|
||||
## Testing Differences
|
||||
|
||||
### BMM Testing (TEA)
|
||||
- Web-focused (Playwright, Cypress)
|
||||
- API testing
|
||||
- E2E for web applications
|
||||
|
||||
### BMGD Testing (Game QA)
|
||||
- Engine-specific frameworks (Unity, Unreal, Godot)
|
||||
- Gameplay testing
|
||||
- Performance profiling
|
||||
- Playtest planning
|
||||
- Balance validation
|
||||
|
||||
## Production Workflow
|
||||
|
||||
BMGD production workflows **inherit from BMM** and add game-specific:
|
||||
- Checklists
|
||||
- Templates
|
||||
- Quality gates
|
||||
- Engine-specific considerations
|
||||
|
||||
This means you get all of BMM's implementation structure plus game-specific enhancements.
|
||||
|
||||
## When to Use Each
|
||||
|
||||
### Use BMM when:
|
||||
- Building web applications
|
||||
- Creating APIs and services
|
||||
- Developing mobile apps (non-game)
|
||||
- Any general software project
|
||||
|
||||
### Use BMGD when:
|
||||
- Building video games
|
||||
- Creating interactive experiences
|
||||
- Game prototyping
|
||||
- Game jams
|
||||
|
|
@ -1,447 +0,0 @@
|
|||
---
|
||||
title: "BMGD Game Types Guide"
|
||||
---
|
||||
|
||||
Reference for selecting and using BMGD's 24 supported game type templates.
|
||||
|
||||
## Overview
|
||||
|
||||
When creating a GDD, BMGD offers game type templates that provide genre-specific sections. This ensures your design document covers mechanics and systems relevant to your game's genre.
|
||||
|
||||
## Supported Game Types
|
||||
|
||||
### Action & Combat
|
||||
|
||||
#### Action Platformer
|
||||
|
||||
**Tags:** action, platformer, combat, movement
|
||||
|
||||
Side-scrolling or 3D platforming with combat mechanics. Think Hollow Knight, Celeste with combat, or Mega Man.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Movement systems (jumps, dashes, wall mechanics)
|
||||
- Combat mechanics (melee/ranged, combos)
|
||||
- Level design patterns
|
||||
- Boss design
|
||||
|
||||
#### Shooter
|
||||
|
||||
**Tags:** shooter, combat, aiming, fps, tps
|
||||
|
||||
Projectile combat with aiming mechanics. Covers FPS, TPS, and arena shooters.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Weapon systems
|
||||
- Aiming and accuracy
|
||||
- Enemy AI patterns
|
||||
- Level/arena design
|
||||
- Multiplayer considerations
|
||||
|
||||
#### Fighting
|
||||
|
||||
**Tags:** fighting, combat, competitive, combos, pvp
|
||||
|
||||
1v1 combat with combos and frame data. Traditional fighters and platform fighters.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Frame data systems
|
||||
- Combo mechanics
|
||||
- Character movesets
|
||||
- Competitive balance
|
||||
- Netcode requirements
|
||||
|
||||
### Strategy & Tactics
|
||||
|
||||
#### Strategy
|
||||
|
||||
**Tags:** strategy, tactics, resources, planning
|
||||
|
||||
Resource management with tactical decisions. RTS, 4X, and grand strategy.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Resource systems
|
||||
- Unit/building design
|
||||
- AI opponent behavior
|
||||
- Map/scenario design
|
||||
- Victory conditions
|
||||
|
||||
#### Turn-Based Tactics
|
||||
|
||||
**Tags:** tactics, turn-based, grid, positioning
|
||||
|
||||
Grid-based movement with turn order. XCOM-likes and tactical RPGs.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Grid and movement systems
|
||||
- Turn order mechanics
|
||||
- Cover and positioning
|
||||
- Unit progression
|
||||
- Procedural mission generation
|
||||
|
||||
#### Tower Defense
|
||||
|
||||
**Tags:** tower-defense, waves, placement, strategy
|
||||
|
||||
Wave-based defense with tower placement.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Tower types and upgrades
|
||||
- Wave design and pacing
|
||||
- Economy systems
|
||||
- Map design patterns
|
||||
- Meta-progression
|
||||
|
||||
### RPG & Progression
|
||||
|
||||
#### RPG
|
||||
|
||||
**Tags:** rpg, stats, inventory, quests, narrative
|
||||
|
||||
Character progression with stats, inventory, and quests.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Character stats and leveling
|
||||
- Inventory and equipment
|
||||
- Quest system design
|
||||
- Combat system (action/turn-based)
|
||||
- Skill trees and builds
|
||||
|
||||
#### Roguelike
|
||||
|
||||
**Tags:** roguelike, procedural, permadeath, runs
|
||||
|
||||
Procedural generation with permadeath and run-based progression.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Procedural generation rules
|
||||
- Permadeath and persistence
|
||||
- Run structure and pacing
|
||||
- Item/ability synergies
|
||||
- Meta-progression systems
|
||||
|
||||
#### Metroidvania
|
||||
|
||||
**Tags:** metroidvania, exploration, abilities, interconnected
|
||||
|
||||
Interconnected world with ability gating.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- World map connectivity
|
||||
- Ability gating design
|
||||
- Backtracking flow
|
||||
- Secret and collectible placement
|
||||
- Power-up progression
|
||||
|
||||
### Narrative & Story
|
||||
|
||||
#### Adventure
|
||||
|
||||
**Tags:** adventure, narrative, exploration, story
|
||||
|
||||
Story-driven exploration and narrative. Point-and-click and narrative adventures.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Puzzle design
|
||||
- Narrative delivery
|
||||
- Exploration mechanics
|
||||
- Dialogue systems
|
||||
- Story branching
|
||||
|
||||
#### Visual Novel
|
||||
|
||||
**Tags:** visual-novel, narrative, choices, story
|
||||
|
||||
Narrative choices with branching story.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Branching narrative structure
|
||||
- Choice and consequence
|
||||
- Character routes
|
||||
- UI/presentation
|
||||
- Save/load states
|
||||
|
||||
#### Text-Based
|
||||
|
||||
**Tags:** text, parser, interactive-fiction, mud
|
||||
|
||||
Text input/output games. Parser games, choice-based IF, MUDs.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Parser or choice systems
|
||||
- World model
|
||||
- Narrative structure
|
||||
- Text presentation
|
||||
- Save state management
|
||||
|
||||
### Simulation & Management
|
||||
|
||||
#### Simulation
|
||||
|
||||
**Tags:** simulation, management, sandbox, systems
|
||||
|
||||
Realistic systems with management and building. Includes tycoons and sim games.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Core simulation loops
|
||||
- Economy modeling
|
||||
- AI agents/citizens
|
||||
- Building/construction
|
||||
- Failure states
|
||||
|
||||
#### Sandbox
|
||||
|
||||
**Tags:** sandbox, creative, building, freedom
|
||||
|
||||
Creative freedom with building and minimal objectives.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Creation tools
|
||||
- Physics/interaction systems
|
||||
- Persistence and saving
|
||||
- Sharing/community features
|
||||
- Optional objectives
|
||||
|
||||
### Sports & Racing
|
||||
|
||||
#### Racing
|
||||
|
||||
**Tags:** racing, vehicles, tracks, speed
|
||||
|
||||
Vehicle control with tracks and lap times.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Vehicle physics model
|
||||
- Track design
|
||||
- AI opponents
|
||||
- Progression/career mode
|
||||
- Multiplayer racing
|
||||
|
||||
#### Sports
|
||||
|
||||
**Tags:** sports, teams, realistic, physics
|
||||
|
||||
Team-based or individual sports simulation.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Sport-specific rules
|
||||
- Player/team management
|
||||
- AI opponent behavior
|
||||
- Season/career modes
|
||||
- Multiplayer modes
|
||||
|
||||
### Multiplayer
|
||||
|
||||
#### MOBA
|
||||
|
||||
**Tags:** moba, multiplayer, pvp, heroes, lanes
|
||||
|
||||
Multiplayer team battles with hero selection.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Hero/champion design
|
||||
- Lane and map design
|
||||
- Team composition
|
||||
- Matchmaking
|
||||
- Economy (gold/items)
|
||||
|
||||
#### Party Game
|
||||
|
||||
**Tags:** party, multiplayer, minigames, casual
|
||||
|
||||
Local multiplayer with minigames.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Minigame design patterns
|
||||
- Controller support
|
||||
- Round/game structure
|
||||
- Scoring systems
|
||||
- Player count flexibility
|
||||
|
||||
### Horror & Survival
|
||||
|
||||
#### Survival
|
||||
|
||||
**Tags:** survival, crafting, resources, danger
|
||||
|
||||
Resource gathering with crafting and persistent threats.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Resource gathering
|
||||
- Crafting systems
|
||||
- Hunger/health/needs
|
||||
- Threat systems
|
||||
- Base building
|
||||
|
||||
#### Horror
|
||||
|
||||
**Tags:** horror, atmosphere, tension, fear
|
||||
|
||||
Atmosphere and tension with limited resources.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Fear mechanics
|
||||
- Resource scarcity
|
||||
- Sound design
|
||||
- Lighting and visibility
|
||||
- Enemy/threat design
|
||||
|
||||
### Casual & Progression
|
||||
|
||||
#### Puzzle
|
||||
|
||||
**Tags:** puzzle, logic, cerebral
|
||||
|
||||
Logic-based challenges and problem-solving.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Puzzle mechanics
|
||||
- Difficulty progression
|
||||
- Hint systems
|
||||
- Level structure
|
||||
- Scoring/rating
|
||||
|
||||
#### Idle/Incremental
|
||||
|
||||
**Tags:** idle, incremental, automation, progression
|
||||
|
||||
Passive progression with upgrades and automation.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Core loop design
|
||||
- Prestige systems
|
||||
- Automation unlocks
|
||||
- Number scaling
|
||||
- Offline progress
|
||||
|
||||
#### Card Game
|
||||
|
||||
**Tags:** card, deck-building, strategy, turns
|
||||
|
||||
Deck building with card mechanics.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Card design framework
|
||||
- Deck building rules
|
||||
- Mana/resource systems
|
||||
- Rarity and collection
|
||||
- Competitive balance
|
||||
|
||||
### Rhythm
|
||||
|
||||
#### Rhythm
|
||||
|
||||
**Tags:** rhythm, music, timing, beats
|
||||
|
||||
Music synchronization with timing-based gameplay.
|
||||
|
||||
**GDD sections added:**
|
||||
|
||||
- Note/beat mapping
|
||||
- Scoring systems
|
||||
- Difficulty levels
|
||||
- Music licensing
|
||||
- Input methods
|
||||
|
||||
## Hybrid Game Types
|
||||
|
||||
Many games combine multiple genres. BMGD supports hybrid selection:
|
||||
|
||||
### Examples
|
||||
|
||||
**Action RPG** = Action Platformer + RPG
|
||||
|
||||
- Movement and combat systems from Action Platformer
|
||||
- Progression and stats from RPG
|
||||
|
||||
**Survival Horror** = Survival + Horror
|
||||
|
||||
- Resource and crafting from Survival
|
||||
- Atmosphere and fear from Horror
|
||||
|
||||
**Roguelike Deckbuilder** = Roguelike + Card Game
|
||||
|
||||
- Run structure from Roguelike
|
||||
- Card mechanics from Card Game
|
||||
|
||||
### How to Use Hybrids
|
||||
|
||||
During GDD creation, select multiple game types when prompted:
|
||||
|
||||
```
|
||||
Agent: What game type best describes your game?
|
||||
You: It's a roguelike with card game combat
|
||||
Agent: I'll include sections for both Roguelike and Card Game...
|
||||
```
|
||||
|
||||
## Game Type Selection Tips
|
||||
|
||||
### 1. Start with Core Fantasy
|
||||
|
||||
What does the player primarily DO in your game?
|
||||
|
||||
- Run and jump? → Platformer types
|
||||
- Build and manage? → Simulation types
|
||||
- Fight enemies? → Combat types
|
||||
- Make choices? → Narrative types
|
||||
|
||||
### 2. Consider Your Loop
|
||||
|
||||
What's the core gameplay loop?
|
||||
|
||||
- Session-based runs? → Roguelike
|
||||
- Long-term progression? → RPG
|
||||
- Quick matches? → Multiplayer types
|
||||
- Creative expression? → Sandbox
|
||||
|
||||
### 3. Don't Over-Combine
|
||||
|
||||
2-3 game types maximum. More than that usually means your design isn't focused enough.
|
||||
|
||||
### 4. Primary vs Secondary
|
||||
|
||||
One type should be primary (most gameplay time). Others add flavor:
|
||||
|
||||
- **Primary:** Platformer (core movement and exploration)
|
||||
- **Secondary:** Metroidvania (ability gating structure)
|
||||
|
||||
## GDD Section Mapping
|
||||
|
||||
When you select a game type, BMGD adds these GDD sections:
|
||||
|
||||
| Game Type | Key Sections Added |
|
||||
| ----------------- | -------------------------------------- |
|
||||
| Action Platformer | Movement, Combat, Level Design |
|
||||
| RPG | Stats, Inventory, Quests |
|
||||
| Roguelike | Procedural Gen, Runs, Meta-Progression |
|
||||
| Narrative | Story Structure, Dialogue, Branching |
|
||||
| Multiplayer | Matchmaking, Netcode, Balance |
|
||||
| Simulation | Systems, Economy, AI |
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **[Quick Start Guide](/docs/tutorials/getting-started/quick-start-bmgd.md)** - Get started with BMGD
|
||||
- **[Workflows Guide](/docs/reference/workflows/bmgd-workflows.md)** - GDD workflow details
|
||||
- **[Glossary](/docs/reference/glossary/index.md)** - Game development terminology
|
||||
|
|
@ -1,70 +0,0 @@
|
|||
---
|
||||
title: "BMGD - Game Development Module"
|
||||
description: AI-powered workflows for game design and development with BMGD
|
||||
---
|
||||
|
||||
Complete guides for the BMad Game Development Module (BMGD) — AI-powered workflows for game design and development that adapt to your project's needs.
|
||||
|
||||
## Getting Started
|
||||
|
||||
**New to BMGD?** Start here:
|
||||
|
||||
- **[Quick Start Guide](/docs/tutorials/getting-started/quick-start-bmgd.md)** - Get started building your first game
|
||||
- Installation and setup
|
||||
- Understanding the game development phases
|
||||
- Running your first workflows
|
||||
- Agent-based development flow
|
||||
|
||||
:::tip[Quick Path]
|
||||
Install BMGD module → Game Brief → GDD → Architecture → Build
|
||||
:::
|
||||
|
||||
## Core Documentation
|
||||
|
||||
- **[Game Types Guide](/docs/explanation/game-dev/game-types.md)** - Selecting and using game type templates (24 supported types)
|
||||
- **[BMGD vs BMM](/docs/explanation/game-dev/bmgd-vs-bmm.md)** - Understanding the differences
|
||||
|
||||
## Game Development Phases
|
||||
|
||||
BMGD follows four phases aligned with game development:
|
||||
|
||||
### Phase 1: Preproduction
|
||||
- **Brainstorm Game** - Ideation with game-specific techniques
|
||||
- **Game Brief** - Capture vision, market, and fundamentals
|
||||
|
||||
### Phase 2: Design
|
||||
- **GDD (Game Design Document)** - Comprehensive game design
|
||||
- **Narrative Design** - Story, characters, world (for story-driven games)
|
||||
|
||||
### Phase 3: Technical
|
||||
- **Game Architecture** - Engine, systems, patterns, structure
|
||||
|
||||
### Phase 4: Production
|
||||
- **Sprint Planning** - Epic and story management
|
||||
- **Story Development** - Implementation workflow
|
||||
- **Code Review** - Quality assurance
|
||||
- **Testing** - Automated tests, playtesting, performance
|
||||
- **Retrospective** - Continuous improvement
|
||||
|
||||
## Choose Your Path
|
||||
|
||||
### I need to...
|
||||
|
||||
**Start a new game project**
|
||||
→ Start with [Quick Start Guide](/docs/tutorials/getting-started/quick-start-bmgd.md)
|
||||
→ Run `brainstorm-game` for ideation
|
||||
→ Create a Game Brief with `create-brief`
|
||||
|
||||
**Design my game**
|
||||
→ Create a GDD with `create-gdd`
|
||||
→ If story-heavy, add Narrative Design with `create-narrative`
|
||||
|
||||
**Plan the technical architecture**
|
||||
→ Run `create-architecture` with the Game Architect
|
||||
|
||||
**Build my game**
|
||||
→ Use Phase 4 production workflows
|
||||
→ Follow the sprint-based development cycle
|
||||
|
||||
**Quickly test an idea**
|
||||
→ Use [Quick-Flow](/docs/how-to/workflows/bmgd-quick-flow.md) for rapid prototyping
|
||||
|
|
@ -1,106 +1,333 @@
|
|||
---
|
||||
title: "Facilitation Over Generation"
|
||||
description: Understanding CIS's facilitation-first approach to creative work
|
||||
description: Understanding a facilitation-first approach to AI workflows and creative collaboration
|
||||
---
|
||||
|
||||
BMAD workflows take a fundamentally different approach from typical AI Prompts you will find. Instead of generating solutions directly, workflows act as facilitators who guide you through discovery processes, helping you arrive at insights and decisions yourself.
|
||||
|
||||
The Creative Intelligence Suite (CIS) takes a fundamentally different approach from typical AI tools. Instead of generating solutions directly, CIS agents act as master facilitators who guide you to discover insights yourself.
|
||||
This has always been the magic of the BMad method - if you have used it you have felt it intuitively and noticed a difference - and it is through this document I want to really explain what this is. This is fundamental to understand if you are also interested in producing your own workflows or improving upon existing ones in any BMad related module.
|
||||
|
||||
## The Problem with Generation
|
||||
|
||||
Traditional AI approaches to creative work:
|
||||
Traditional AI approaches to creative and knowledge work typically follow this pattern:
|
||||
|
||||
:::note[Generation Example]
|
||||
**User:** "Give me marketing ideas"
|
||||
**User:** "I need a comprehensive UX design for my e-commerce app"
|
||||
|
||||
**AI:** "Here are 10 marketing ideas..."
|
||||
**AI:** [Immediately generates] "Here's a complete UX specification with modern dark theme, card-based product grid, left navigation, and checkout flow..."
|
||||
:::
|
||||
|
||||
This approach:
|
||||
- Produces generic, predictable outputs
|
||||
- Removes human ownership of ideas
|
||||
- Misses context and nuance
|
||||
- Limits creative exploration
|
||||
- Produces generic, predictable outputs that may not fit your actual needs
|
||||
- Removes your ownership and understanding of the solution
|
||||
- Misses critical context, constraints, and nuance
|
||||
- Limits creative exploration to the AI's training data patterns
|
||||
- Creates deliverables you can't explain or iterate on effectively
|
||||
|
||||
## The Facilitation Approach
|
||||
|
||||
CIS agents use strategic questioning:
|
||||
Facilitative workflows use strategic questioning and guided discovery:
|
||||
|
||||
:::note[Facilitation Example]
|
||||
**User:** "I need marketing ideas"
|
||||
:::note[Facilitation Example from UX Design Workflow]
|
||||
**User:** "I need a comprehensive UX design for my e-commerce app"
|
||||
|
||||
**CIS:** "What makes your customers choose you over alternatives? What's the one thing they always mention?"
|
||||
**Workflow:** "Welcome! I'll act as your UX facilitator to help you discover the right approach. First, let me understand what documents you already have—do you have a product brief, PRD, or any research?"
|
||||
|
||||
**User:** "They say our support is exceptional"
|
||||
**User:** "I have a product brief and some user research"
|
||||
|
||||
**CIS:** "Interesting! How might you make that exceptional support visible before they become customers?"
|
||||
**Workflow:** "Great! Let me load those. [Loads documents] Now, what are the primary user goals for your e-commerce experience? What should users be able to accomplish?"
|
||||
|
||||
**User:** "They need to browse products quickly and complete purchases without friction"
|
||||
|
||||
**Workflow:** "Interesting! And what makes your e-commerce situation unique? What are the specific challenges or constraints you're working with?"
|
||||
:::
|
||||
|
||||
This approach:
|
||||
- Draws out insights already within you
|
||||
- Maintains human ownership of ideas
|
||||
- Captures context and nuance
|
||||
- Enables deeper creative exploration
|
||||
- Draws out insights and expertise already within you
|
||||
- Maintains your ownership and understanding of decisions
|
||||
- Captures your specific context, constraints, and goals
|
||||
- Enables deeper exploration of your unique situation
|
||||
- Creates outputs you can confidently explain and iterate on
|
||||
|
||||
## Real-World Examples from BMAD Workflows
|
||||
|
||||
### Brainstorming Workflow (from core module)
|
||||
|
||||
The brainstorming workflow demonstrates pure facilitation through its entire journey:
|
||||
|
||||
**Session Setup:**
|
||||
```
|
||||
"Welcome! I'm excited to facilitate your brainstorming session. I'll guide you
|
||||
through proven creativity techniques to generate innovative ideas.
|
||||
|
||||
**What are we brainstorming about?** (The central topic or challenge)
|
||||
**What specific outcomes are you hoping for?** (Types of ideas, solutions, or insights)
|
||||
```
|
||||
|
||||
**Technique Selection - Offering Options:**
|
||||
```
|
||||
"Ready to explore technique approaches?
|
||||
[1] User-Selected Techniques - Browse our complete technique library
|
||||
[2] AI-Recommended Techniques - Get customized suggestions based on your goals
|
||||
[3] Random Technique Selection - Discover unexpected creative methods
|
||||
[4] Progressive Technique Flow - Start broad, then systematically narrow focus
|
||||
|
||||
Which approach appeals to you most?"
|
||||
```
|
||||
|
||||
**Technique Execution - Interactive Coaching:**
|
||||
The workflow doesn't generate ideas—it coaches you through techniques with genuine back-and-forth dialogue:
|
||||
|
||||
```
|
||||
"Let's start with: What if you could remove all practical constraints?
|
||||
|
||||
I'm not just looking for a quick answer - I want to explore this together.
|
||||
What immediately comes to mind? Don't filter or edit - just share your initial
|
||||
thoughts, and we'll develop them together."
|
||||
|
||||
[User responds]
|
||||
|
||||
"That's interesting! Tell me more about [specific aspect you mentioned].
|
||||
What would that look like in practice? How does that connect to your core goal?"
|
||||
```
|
||||
|
||||
**Key facilitation behaviors:**
|
||||
- Aims for 100+ ideas before suggesting organization
|
||||
- Asks "Continue exploring?" or "Move to next technique?"—user controls pace
|
||||
- Uses anti-bias protocols to force thinking in new directions every 10 ideas
|
||||
- Builds on user's ideas with genuine creative contributions
|
||||
- Keeps user in "generative exploration mode" as long as possible
|
||||
|
||||
**Organization - Collaborative Synthesis:**
|
||||
```
|
||||
"Outstanding creative work! You've generated an incredible range of ideas.
|
||||
Now let's organize these creative gems and identify your most promising opportunities.
|
||||
|
||||
I'm analyzing all your generated ideas to identify natural themes and patterns.
|
||||
**Emerging Themes I'm Identifying:**
|
||||
- Theme 1: [Name] - Ideas: [list] - Pattern: [connection]
|
||||
- Theme 2: [Name] - Ideas: [list] - Pattern: [connection]
|
||||
|
||||
Which themes or specific ideas stand out to you as most valuable?"
|
||||
```
|
||||
|
||||
Result: A comprehensive brainstorming session document with **your** ideas, organized by **your** priorities, with **your** action plans.
|
||||
|
||||
### Create UX Design Workflow (from BMM method)
|
||||
|
||||
The UX design workflow facilitates a 14-step journey from project understanding to complete UX specification—**never making design decisions for you**.
|
||||
|
||||
**Step 1: Document Discovery (Collaborative Setup)**
|
||||
```
|
||||
"Welcome! I've set up your UX design workspace.
|
||||
|
||||
**Documents Found:**
|
||||
- PRD: product-requirements.md
|
||||
- Product brief: brief.md
|
||||
|
||||
**Files loaded:** [lists specific files]
|
||||
|
||||
Do you have any other documents you'd like me to include, or shall we continue?"
|
||||
```
|
||||
|
||||
**Step 2: Project Understanding (Discovery Questions)**
|
||||
```
|
||||
"Based on the project documentation, let me confirm what I'm understanding...
|
||||
|
||||
**From the documents:** [summary of key insights]
|
||||
**Target Users:** [summary from documents]
|
||||
**Key Features/Goals:** [summary from documents]
|
||||
|
||||
Does this match your understanding? Are there any corrections or additions?"
|
||||
```
|
||||
|
||||
Then it dives deeper with targeted questions:
|
||||
```
|
||||
"Let me understand your users better to inform the UX design:
|
||||
|
||||
**User Context Questions:**
|
||||
- What problem are users trying to solve?
|
||||
- What frustrates them with current solutions?
|
||||
- What would make them say 'this is exactly what I needed'?"
|
||||
```
|
||||
|
||||
**Step 3: Core Experience Definition (Guiding Insights)**
|
||||
```
|
||||
"Now let's dig into the heart of the user experience.
|
||||
|
||||
**Core Experience Questions:**
|
||||
- What's the ONE thing users will do most frequently?
|
||||
- What user action is absolutely critical to get right?
|
||||
- What should be completely effortless for users?
|
||||
- If we nail one interaction, everything else follows - what is it?
|
||||
|
||||
Think about the core loop or primary action that defines your product's value."
|
||||
```
|
||||
|
||||
**Step 4: Emotional Response (Feelings-Based Design)**
|
||||
```
|
||||
"Now let's think about how your product should make users feel.
|
||||
|
||||
**Emotional Response Questions:**
|
||||
- What should users FEEL when using this product?
|
||||
- What emotion would make them tell a friend about this?
|
||||
- How should users feel after accomplishing their primary goal?
|
||||
|
||||
Common emotional goals: Empowered and in control? Delighted and surprised?
|
||||
Efficient and productive? Creative and inspired?"
|
||||
```
|
||||
|
||||
**Step 5: Pattern Inspiration (Learning from Examples)**
|
||||
```
|
||||
"Let's learn from products your users already love and use regularly.
|
||||
|
||||
**Inspiration Questions:**
|
||||
- Name 2-3 apps your target users already love and USE frequently
|
||||
- For each one, what do they do well from a UX perspective?
|
||||
- What makes the experience compelling or delightful?
|
||||
|
||||
For each inspiring app, let's analyze their UX success:
|
||||
- What core problem does it solve elegantly?
|
||||
- What makes the onboarding experience effective?
|
||||
- How do they handle navigation and information hierarchy?"
|
||||
```
|
||||
|
||||
**Step 9: Design Directions (Interactive Visual Exploration)**
|
||||
The workflow generates 6-8 HTML mockup variations—but **you choose**:
|
||||
|
||||
```
|
||||
"🎨 Design Direction Mockups Generated!
|
||||
|
||||
I'm creating a comprehensive HTML showcase with 6-8 full-screen mockup variations.
|
||||
Each mockup represents a complete visual direction for your app's look and feel.
|
||||
|
||||
**As you explore the design directions, look for:**
|
||||
✅ Which information hierarchy matches your priorities?
|
||||
✅ Which interaction style fits your core experience?
|
||||
✅ Which visual density feels right for your brand?
|
||||
|
||||
**Which approach resonates most with you?**
|
||||
- Pick a favorite direction as-is
|
||||
- Combine elements from multiple directions
|
||||
- Request modifications to any direction
|
||||
|
||||
Tell me: Which layout feels most intuitive? Which visual weight matches your brand?"
|
||||
```
|
||||
|
||||
**Step 12: UX Patterns (Consistency Through Questions)**
|
||||
```
|
||||
"Let's establish consistency patterns for common situations.
|
||||
|
||||
**Pattern Categories to Define:**
|
||||
- Button hierarchy and actions
|
||||
- Feedback patterns (success, error, warning, info)
|
||||
- Form patterns and validation
|
||||
- Navigation patterns
|
||||
|
||||
Which categories are most critical for your product?
|
||||
|
||||
**For [Critical Pattern Category]:**
|
||||
What should users see/do when they need to [pattern action]?
|
||||
|
||||
**Considerations:**
|
||||
- Visual hierarchy (primary vs. secondary actions)
|
||||
- Feedback mechanisms
|
||||
- Error recovery
|
||||
- Accessibility requirements
|
||||
|
||||
How should your product handle [pattern type] interactions?"
|
||||
```
|
||||
|
||||
**The Result:** A complete, production-ready UX specification document that captures **your** decisions, **your** reasoning, and **your** vision—documented through guided discovery, not generation.
|
||||
|
||||
## Key Principles
|
||||
|
||||
### 1. Questions Over Answers
|
||||
|
||||
CIS agents ask strategic questions rather than providing direct answers. This:
|
||||
- Activates your own creative thinking
|
||||
- Uncovers assumptions
|
||||
- Reveals blind spots
|
||||
- Builds on your domain knowledge
|
||||
Facilitative workflows ask strategic questions rather than providing direct answers. This:
|
||||
- Activates your own creative and analytical thinking
|
||||
- Uncovers assumptions you didn't know you had
|
||||
- Reveals blind spots in your understanding
|
||||
- Builds on your domain expertise and context
|
||||
|
||||
### 2. Energy-Aware Sessions
|
||||
### 2. Multi-Turn Conversation
|
||||
|
||||
CIS monitors engagement and adapts:
|
||||
- Adjusts pace when energy flags
|
||||
- Suggests breaks when needed
|
||||
- Changes techniques to maintain momentum
|
||||
- Recognizes productive vs. unproductive struggle
|
||||
Facilitation uses progressive discovery, not interrogation:
|
||||
- Ask 1-2 questions at a time, not laundry lists
|
||||
- Think about responses before asking follow-ups
|
||||
- Probe to understand deeper, not just collect facts
|
||||
- Use conversation to explore, not just extract
|
||||
|
||||
### 3. Process Trust
|
||||
### 3. Intent-Based Guidance
|
||||
|
||||
CIS uses proven methodologies:
|
||||
- Design Thinking's 5 phases
|
||||
- Structured brainstorming techniques
|
||||
Workflows specify goals and approaches, not exact scripts:
|
||||
- "Guide the user through discovering X" (intent)
|
||||
- NOT "Say exactly: 'What is X?'" (prescriptive)
|
||||
|
||||
This allows the workflow to adapt naturally to your responses while maintaining structured progress.
|
||||
|
||||
### 4. Process Trust
|
||||
|
||||
Facilitative workflows use proven methodologies:
|
||||
- Design Thinking's phases (Empathize, Define, Ideate, Prototype, Test)
|
||||
- Structured brainstorming and creativity techniques
|
||||
- Root cause analysis frameworks
|
||||
- Innovation strategy patterns
|
||||
|
||||
You're not just having a conversation—you're following time-tested creative processes.
|
||||
You're not just having a conversation—you're following time-tested processes adapted to your specific situation.
|
||||
|
||||
### 4. Persona-Driven Engagement
|
||||
### 5. YOU Are the Expert
|
||||
|
||||
Each CIS agent has a distinct personality:
|
||||
- **Carson** - Energetic, encouraging
|
||||
- **Maya** - Jazz-like, improvisational
|
||||
- **Dr. Quinn** - Analytical, methodical
|
||||
- **Victor** - Bold, strategic
|
||||
- **Sophia** - Narrative, imaginative
|
||||
Facilitative workflows operate on a core principle: **you are the expert on your situation**. The workflow brings:
|
||||
- Process expertise (how to think through problems)
|
||||
- Facilitation skills (how to guide exploration)
|
||||
- Technique knowledge (proven methods and frameworks)
|
||||
|
||||
These personas create engaging experiences that maintain creative flow.
|
||||
You bring:
|
||||
- Domain knowledge (your specific field or industry)
|
||||
- Context understanding (your unique situation and constraints)
|
||||
- Decision authority (what will actually work for you)
|
||||
|
||||
## When Generation is Appropriate
|
||||
|
||||
CIS does generate when appropriate:
|
||||
- Synthesizing session outputs
|
||||
- Documenting decisions
|
||||
- Creating structured artifacts
|
||||
- Providing technique examples
|
||||
Facilitative workflows DO generate when appropriate:
|
||||
- Synthesizing and structuring outputs after you've made decisions
|
||||
- Documenting your choices and rationale
|
||||
- Creating structured artifacts based on your input
|
||||
- Providing technique examples or option templates
|
||||
- Formatting and organizing your conclusions
|
||||
|
||||
But the core creative work happens through facilitated discovery.
|
||||
But the **core creative and analytical work** happens through facilitated discovery, not generation.
|
||||
|
||||
## The Distinction: Facilitator vs Generator
|
||||
|
||||
| Facilitative Workflow | Generative AI |
|
||||
| ------------------------------------- | --------------------------------------- |
|
||||
| "What are your goals?" | "Here's the solution" |
|
||||
| Asks 1-2 questions at a time | Produces complete output immediately |
|
||||
| Multiple turns, progressive discovery | Single turn, bulk generation |
|
||||
| "Let me understand your context" | "Here's a generic answer" |
|
||||
| Offers options, you choose | Makes decisions for you |
|
||||
| Documents YOUR reasoning | No reasoning visible |
|
||||
| You can explain every decision | You can't explain why choices were made |
|
||||
| Ownership and understanding | Outputs feel alien |
|
||||
|
||||
## Benefits
|
||||
|
||||
### For Individuals
|
||||
- Deeper insights than pure generation
|
||||
- Ownership of creative outputs
|
||||
- Skill development in creative thinking
|
||||
- More memorable and actionable ideas
|
||||
- **Deeper insights** than pure generation—ideas connect to your actual knowledge
|
||||
- **Full ownership** of creative outputs and decisions
|
||||
- **Skill development** in structured thinking and problem-solving
|
||||
- **More memorable and actionable** results—you understand the "why"
|
||||
|
||||
### For Teams
|
||||
- Shared creative experience
|
||||
- Aligned understanding
|
||||
- Documented rationale
|
||||
- Stronger buy-in to outcomes
|
||||
- **Shared creative experience** building alignment and trust
|
||||
- **Aligned understanding** through documented exploration
|
||||
- **Documented rationale** for future reference and onboarding
|
||||
- **Stronger buy-in** to outcomes because everyone participated in discovery
|
||||
|
||||
### For Implementation
|
||||
- **Outputs match reality** because they emerged from your actual constraints
|
||||
- **Easier iteration** because you understand the reasoning behind choices
|
||||
- **Confident implementation** because you can defend every decision
|
||||
- **Reduced rework** because facilitation catches issues early
|
||||
|
|
|
|||
|
|
@ -0,0 +1,710 @@
|
|||
---
|
||||
title: "TEA Engagement Models Explained"
|
||||
description: Understanding the five ways to use TEA - from standalone to full BMad Method integration
|
||||
---
|
||||
|
||||
# TEA Engagement Models Explained
|
||||
|
||||
TEA is optional and flexible. There are five valid ways to engage with TEA - choose intentionally based on your project needs and methodology.
|
||||
|
||||
## Overview
|
||||
|
||||
**TEA is not mandatory.** Pick the engagement model that fits your context:
|
||||
|
||||
1. **No TEA** - Skip all TEA workflows, use existing testing approach
|
||||
2. **TEA Solo** - Use TEA standalone without BMad Method
|
||||
3. **TEA Lite** - Beginner approach using just `*automate`
|
||||
4. **TEA Integrated (Greenfield)** - Full BMad Method integration from scratch
|
||||
5. **TEA Integrated (Brownfield)** - Full BMad Method integration with existing code
|
||||
|
||||
## The Problem
|
||||
|
||||
### One-Size-Fits-All Doesn't Work
|
||||
|
||||
**Traditional testing tools force one approach:**
|
||||
- Must use entire framework
|
||||
- All-or-nothing adoption
|
||||
- No flexibility for different project types
|
||||
- Teams abandon tool if it doesn't fit
|
||||
|
||||
**TEA recognizes:**
|
||||
- Different projects have different needs
|
||||
- Different teams have different maturity levels
|
||||
- Different contexts require different approaches
|
||||
- Flexibility increases adoption
|
||||
|
||||
## The Five Engagement Models
|
||||
|
||||
### Model 1: No TEA
|
||||
|
||||
**What:** Skip all TEA workflows, use your existing testing approach.
|
||||
|
||||
**When to Use:**
|
||||
- Team has established testing practices
|
||||
- Quality is already high
|
||||
- Testing tools already in place
|
||||
- TEA doesn't add value
|
||||
|
||||
**What You Miss:**
|
||||
- Risk-based test planning
|
||||
- Systematic quality review
|
||||
- Gate decisions with evidence
|
||||
- Knowledge base patterns
|
||||
|
||||
**What You Keep:**
|
||||
- Full control
|
||||
- Existing tools
|
||||
- Team expertise
|
||||
- No learning curve
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Your team:
|
||||
- 10-year veteran QA team
|
||||
- Established testing practices
|
||||
- High-quality test suite
|
||||
- No problems to solve
|
||||
|
||||
Decision: Skip TEA, keep what works
|
||||
```
|
||||
|
||||
**Verdict:** Valid choice if existing approach works.
|
||||
|
||||
---
|
||||
|
||||
### Model 2: TEA Solo
|
||||
|
||||
**What:** Use TEA workflows standalone without full BMad Method integration.
|
||||
|
||||
**When to Use:**
|
||||
- Non-BMad projects
|
||||
- Want TEA's quality operating model only
|
||||
- Don't need full planning workflow
|
||||
- Bring your own requirements
|
||||
|
||||
**Typical Sequence:**
|
||||
```
|
||||
1. *test-design (system or epic)
|
||||
2. *atdd or *automate
|
||||
3. *test-review (optional)
|
||||
4. *trace (coverage + gate decision)
|
||||
```
|
||||
|
||||
**You Bring:**
|
||||
- Requirements (user stories, acceptance criteria)
|
||||
- Development environment
|
||||
- Project context
|
||||
|
||||
**TEA Provides:**
|
||||
- Risk-based test planning (`*test-design`)
|
||||
- Test generation (`*atdd`, `*automate`)
|
||||
- Quality review (`*test-review`)
|
||||
- Coverage traceability (`*trace`)
|
||||
|
||||
**Optional:**
|
||||
- Framework setup (`*framework`) if needed
|
||||
- CI configuration (`*ci`) if needed
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Your project:
|
||||
- Using Scrum (not BMad Method)
|
||||
- Jira for story management
|
||||
- Need better test strategy
|
||||
|
||||
Workflow:
|
||||
1. Export stories from Jira
|
||||
2. Run *test-design on epic
|
||||
3. Run *atdd for each story
|
||||
4. Implement features
|
||||
5. Run *trace for coverage
|
||||
```
|
||||
|
||||
**Verdict:** Best for teams wanting TEA benefits without BMad Method commitment.
|
||||
|
||||
---
|
||||
|
||||
### Model 3: TEA Lite
|
||||
|
||||
**What:** Beginner approach using just `*automate` to test existing features.
|
||||
|
||||
**When to Use:**
|
||||
- Learning TEA fundamentals
|
||||
- Want quick results
|
||||
- Testing existing application
|
||||
- No time for full methodology
|
||||
|
||||
**Workflow:**
|
||||
```
|
||||
1. *framework (setup test infrastructure)
|
||||
2. *test-design (optional, risk assessment)
|
||||
3. *automate (generate tests for existing features)
|
||||
4. Run tests (they pass immediately)
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Beginner developer:
|
||||
- Never used TEA before
|
||||
- Want to add tests to existing app
|
||||
- 30 minutes available
|
||||
|
||||
Steps:
|
||||
1. Run *framework
|
||||
2. Run *automate on TodoMVC demo
|
||||
3. Tests generated and passing
|
||||
4. Learn TEA basics
|
||||
```
|
||||
|
||||
**What You Get:**
|
||||
- Working test framework
|
||||
- Passing tests for existing features
|
||||
- Learning experience
|
||||
- Foundation to expand
|
||||
|
||||
**What You Miss:**
|
||||
- TDD workflow (ATDD)
|
||||
- Risk-based planning (test-design depth)
|
||||
- Quality gates (trace Phase 2)
|
||||
- Full TEA capabilities
|
||||
|
||||
**Verdict:** Perfect entry point for beginners.
|
||||
|
||||
---
|
||||
|
||||
### Model 4: TEA Integrated (Greenfield)
|
||||
|
||||
**What:** Full BMad Method integration with TEA workflows across all phases.
|
||||
|
||||
**When to Use:**
|
||||
- New projects starting from scratch
|
||||
- Using BMad Method or Enterprise track
|
||||
- Want complete quality operating model
|
||||
- Testing is critical to success
|
||||
|
||||
**Lifecycle:**
|
||||
|
||||
**Phase 2: Planning**
|
||||
- PM creates PRD with NFRs
|
||||
- (Optional) TEA runs `*nfr-assess` (Enterprise only)
|
||||
|
||||
**Phase 3: Solutioning**
|
||||
- Architect creates architecture
|
||||
- TEA runs `*test-design` (system-level) → testability review
|
||||
- TEA runs `*framework` → test infrastructure
|
||||
- TEA runs `*ci` → CI/CD pipeline
|
||||
- Architect runs `*implementation-readiness` (fed by test design)
|
||||
|
||||
**Phase 4: Implementation (Per Epic)**
|
||||
- SM runs `*sprint-planning`
|
||||
- TEA runs `*test-design` (epic-level) → risk assessment for THIS epic
|
||||
- SM creates stories
|
||||
- (Optional) TEA runs `*atdd` → failing tests before dev
|
||||
- DEV implements story
|
||||
- TEA runs `*automate` → expand coverage
|
||||
- (Optional) TEA runs `*test-review` → quality audit
|
||||
- TEA runs `*trace` Phase 1 → refresh coverage
|
||||
|
||||
**Release Gate:**
|
||||
- (Optional) TEA runs `*test-review` → final audit
|
||||
- (Optional) TEA runs `*nfr-assess` → validate NFRs
|
||||
- TEA runs `*trace` Phase 2 → gate decision (PASS/CONCERNS/FAIL/WAIVED)
|
||||
|
||||
**What You Get:**
|
||||
- Complete quality operating model
|
||||
- Systematic test planning
|
||||
- Risk-based prioritization
|
||||
- Evidence-based gate decisions
|
||||
- Consistent patterns across epics
|
||||
|
||||
**Example:**
|
||||
```
|
||||
New SaaS product:
|
||||
- 50 stories across 8 epics
|
||||
- Security critical
|
||||
- Need quality gates
|
||||
|
||||
Workflow:
|
||||
- Phase 2: Define NFRs in PRD
|
||||
- Phase 3: Architecture → test design → framework → CI
|
||||
- Phase 4: Per epic: test design → ATDD → dev → automate → review → trace
|
||||
- Gate: NFR assess → trace Phase 2 → decision
|
||||
```
|
||||
|
||||
**Verdict:** Most comprehensive TEA usage, best for structured teams.
|
||||
|
||||
---
|
||||
|
||||
### Model 5: TEA Integrated (Brownfield)
|
||||
|
||||
**What:** Full BMad Method integration with TEA for existing codebases.
|
||||
|
||||
**When to Use:**
|
||||
- Existing codebase with legacy tests
|
||||
- Want to improve test quality incrementally
|
||||
- Adding features to existing application
|
||||
- Need to establish coverage baseline
|
||||
|
||||
**Differences from Greenfield:**
|
||||
|
||||
**Phase 0: Documentation (if needed)**
|
||||
```
|
||||
- Run *document-project
|
||||
- Create baseline documentation
|
||||
```
|
||||
|
||||
**Phase 2: Planning**
|
||||
```
|
||||
- TEA runs *trace Phase 1 → establish coverage baseline
|
||||
- PM creates PRD (with existing system context)
|
||||
```
|
||||
|
||||
**Phase 3: Solutioning**
|
||||
```
|
||||
- Architect creates architecture (with brownfield constraints)
|
||||
- TEA runs *test-design (system-level) → testability review
|
||||
- TEA runs *framework (only if modernizing test infra)
|
||||
- TEA runs *ci (update existing CI or create new)
|
||||
```
|
||||
|
||||
**Phase 4: Implementation**
|
||||
```
|
||||
- TEA runs *test-design (epic-level) → focus on REGRESSION HOTSPOTS
|
||||
- Per story: ATDD → dev → automate
|
||||
- TEA runs *test-review → improve legacy test quality
|
||||
- TEA runs *trace Phase 1 → track coverage improvement
|
||||
```
|
||||
|
||||
**Brownfield-Specific:**
|
||||
- Baseline coverage BEFORE planning
|
||||
- Focus on regression hotspots (bug-prone areas)
|
||||
- Incremental quality improvement
|
||||
- Compare coverage to baseline (trending up?)
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Legacy e-commerce platform:
|
||||
- 200 existing tests (30% passing, 70% flaky)
|
||||
- Adding new checkout flow
|
||||
- Want to improve quality
|
||||
|
||||
Workflow:
|
||||
1. Phase 2: *trace baseline → 30% coverage
|
||||
2. Phase 3: *test-design → identify regression risks
|
||||
3. Phase 4: Fix top 20 flaky tests + add tests for new checkout
|
||||
4. Gate: *trace → 60% coverage (2x improvement)
|
||||
```
|
||||
|
||||
**Verdict:** Best for incrementally improving legacy systems.
|
||||
|
||||
---
|
||||
|
||||
## Decision Guide: Which Model?
|
||||
|
||||
### Quick Decision Tree
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'14px'}}}%%
|
||||
flowchart TD
|
||||
Start([Choose TEA Model]) --> BMad{Using<br/>BMad Method?}
|
||||
|
||||
BMad -->|No| NonBMad{Project Type?}
|
||||
NonBMad -->|Learning| Lite[TEA Lite<br/>Just *automate<br/>30 min tutorial]
|
||||
NonBMad -->|Serious Project| Solo[TEA Solo<br/>Standalone workflows<br/>Full capabilities]
|
||||
|
||||
BMad -->|Yes| WantTEA{Want TEA?}
|
||||
WantTEA -->|No| None[No TEA<br/>Use existing approach<br/>Valid choice]
|
||||
WantTEA -->|Yes| ProjectType{New or<br/>Existing?}
|
||||
|
||||
ProjectType -->|New Project| Green[TEA Integrated<br/>Greenfield<br/>Full lifecycle]
|
||||
ProjectType -->|Existing Code| Brown[TEA Integrated<br/>Brownfield<br/>Baseline + improve]
|
||||
|
||||
Green --> Compliance{Compliance<br/>Needs?}
|
||||
Compliance -->|Yes| Enterprise[Enterprise Track<br/>NFR + audit trails]
|
||||
Compliance -->|No| Method[BMad Method Track<br/>Standard quality]
|
||||
|
||||
style Lite fill:#bbdefb,stroke:#1565c0,stroke-width:2px
|
||||
style Solo fill:#c5cae9,stroke:#283593,stroke-width:2px
|
||||
style None fill:#e0e0e0,stroke:#616161,stroke-width:1px
|
||||
style Green fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
|
||||
style Brown fill:#fff9c4,stroke:#f57f17,stroke-width:2px
|
||||
style Enterprise fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px
|
||||
style Method fill:#e1f5fe,stroke:#01579b,stroke-width:2px
|
||||
```
|
||||
|
||||
**Decision Path Examples:**
|
||||
- Learning TEA → TEA Lite (blue)
|
||||
- Non-BMad project → TEA Solo (purple)
|
||||
- BMad + new project + compliance → Enterprise (purple)
|
||||
- BMad + existing code → Brownfield (yellow)
|
||||
- Don't want TEA → No TEA (gray)
|
||||
|
||||
### By Project Type
|
||||
|
||||
| Project Type | Recommended Model | Why |
|
||||
|--------------|------------------|-----|
|
||||
| **New SaaS product** | TEA Integrated (Greenfield) | Full quality operating model from day one |
|
||||
| **Existing app + new feature** | TEA Integrated (Brownfield) | Improve incrementally while adding features |
|
||||
| **Bug fix** | TEA Lite or No TEA | Quick flow, minimal overhead |
|
||||
| **Learning project** | TEA Lite | Learn basics with immediate results |
|
||||
| **Non-BMad enterprise** | TEA Solo | Quality model without full methodology |
|
||||
| **High-quality existing tests** | No TEA | Keep what works |
|
||||
|
||||
### By Team Maturity
|
||||
|
||||
| Team Maturity | Recommended Model | Why |
|
||||
|---------------|------------------|-----|
|
||||
| **Beginners** | TEA Lite → TEA Solo | Learn basics, then expand |
|
||||
| **Intermediate** | TEA Solo or Integrated | Depends on methodology |
|
||||
| **Advanced** | TEA Integrated or No TEA | Full model or existing expertise |
|
||||
|
||||
### By Compliance Needs
|
||||
|
||||
| Compliance | Recommended Model | Why |
|
||||
|------------|------------------|-----|
|
||||
| **None** | Any model | Choose based on project needs |
|
||||
| **Light** (internal audit) | TEA Solo or Integrated | Gate decisions helpful |
|
||||
| **Heavy** (SOC 2, HIPAA) | TEA Integrated (Enterprise) | NFR assessment mandatory |
|
||||
|
||||
## Switching Between Models
|
||||
|
||||
### Can Change Models Mid-Project
|
||||
|
||||
**Scenario:** Start with TEA Lite, expand to TEA Solo
|
||||
|
||||
```
|
||||
Week 1: TEA Lite
|
||||
- Run *framework
|
||||
- Run *automate
|
||||
- Learn basics
|
||||
|
||||
Week 2: Expand to TEA Solo
|
||||
- Add *test-design
|
||||
- Use *atdd for new features
|
||||
- Add *test-review
|
||||
|
||||
Week 3: Continue expanding
|
||||
- Add *trace for coverage
|
||||
- Setup *ci
|
||||
- Full TEA Solo workflow
|
||||
```
|
||||
|
||||
**Benefit:** Start small, expand as comfortable.
|
||||
|
||||
### Can Mix Models
|
||||
|
||||
**Scenario:** TEA Integrated for main features, No TEA for bug fixes
|
||||
|
||||
```
|
||||
Main features (epics):
|
||||
- Use full TEA workflow
|
||||
- Risk assessment, ATDD, quality gates
|
||||
|
||||
Bug fixes:
|
||||
- Skip TEA
|
||||
- Quick Flow + manual testing
|
||||
- Move fast
|
||||
|
||||
Result: TEA where it adds value, skip where it doesn't
|
||||
```
|
||||
|
||||
**Benefit:** Flexible, pragmatic, not dogmatic.
|
||||
|
||||
## Comparison Table
|
||||
|
||||
| Aspect | No TEA | TEA Lite | TEA Solo | Integrated (Green) | Integrated (Brown) |
|
||||
|--------|--------|----------|----------|-------------------|-------------------|
|
||||
| **BMad Required** | No | No | No | Yes | Yes |
|
||||
| **Learning Curve** | None | Low | Medium | High | High |
|
||||
| **Setup Time** | 0 | 30 min | 2 hours | 1 day | 2 days |
|
||||
| **Workflows Used** | 0 | 2-3 | 4-6 | 8 | 8 |
|
||||
| **Test Planning** | Manual | Optional | Yes | Systematic | + Regression focus |
|
||||
| **Quality Gates** | No | No | Optional | Yes | Yes + baseline |
|
||||
| **NFR Assessment** | No | No | No | Optional | Recommended |
|
||||
| **Coverage Tracking** | Manual | No | Optional | Yes | Yes + trending |
|
||||
| **Best For** | Experts | Beginners | Standalone | New projects | Legacy code |
|
||||
|
||||
## Real-World Examples
|
||||
|
||||
### Example 1: Startup (TEA Lite → TEA Integrated)
|
||||
|
||||
**Month 1:** TEA Lite
|
||||
```
|
||||
Team: 3 developers, no QA
|
||||
Testing: Manual only
|
||||
Decision: Start with TEA Lite
|
||||
|
||||
Result:
|
||||
- Run *framework (Playwright setup)
|
||||
- Run *automate (20 tests generated)
|
||||
- Learning TEA basics
|
||||
```
|
||||
|
||||
**Month 3:** TEA Solo
|
||||
```
|
||||
Team: Growing to 5 developers
|
||||
Testing: Automated tests exist
|
||||
Decision: Expand to TEA Solo
|
||||
|
||||
Result:
|
||||
- Add *test-design (risk assessment)
|
||||
- Add *atdd (TDD workflow)
|
||||
- Add *test-review (quality audits)
|
||||
```
|
||||
|
||||
**Month 6:** TEA Integrated
|
||||
```
|
||||
Team: 8 developers, 1 QA
|
||||
Testing: Critical to business
|
||||
Decision: Full BMad Method + TEA Integrated
|
||||
|
||||
Result:
|
||||
- Full lifecycle integration
|
||||
- Quality gates before releases
|
||||
- NFR assessment for enterprise customers
|
||||
```
|
||||
|
||||
### Example 2: Enterprise (TEA Integrated - Brownfield)
|
||||
|
||||
**Project:** Legacy banking application
|
||||
|
||||
**Challenge:**
|
||||
- 500 existing tests (50% flaky)
|
||||
- Adding new features
|
||||
- SOC 2 compliance required
|
||||
|
||||
**Model:** TEA Integrated (Brownfield)
|
||||
|
||||
**Phase 2:**
|
||||
```
|
||||
- *trace baseline → 45% coverage (lots of gaps)
|
||||
- Document current state
|
||||
```
|
||||
|
||||
**Phase 3:**
|
||||
```
|
||||
- *test-design (system) → identify regression hotspots
|
||||
- *framework → modernize test infrastructure
|
||||
- *ci → add selective testing
|
||||
```
|
||||
|
||||
**Phase 4:**
|
||||
```
|
||||
Per epic:
|
||||
- *test-design → focus on regression + new features
|
||||
- Fix top 10 flaky tests
|
||||
- *atdd for new features
|
||||
- *automate for coverage expansion
|
||||
- *test-review → track quality improvement
|
||||
- *trace → compare to baseline
|
||||
```
|
||||
|
||||
**Result after 6 months:**
|
||||
- Coverage: 45% → 85%
|
||||
- Quality score: 52 → 82
|
||||
- Flakiness: 50% → 2%
|
||||
- SOC 2 compliant (traceability + NFR evidence)
|
||||
|
||||
### Example 3: Consultancy (TEA Solo)
|
||||
|
||||
**Context:** Testing consultancy working with multiple clients
|
||||
|
||||
**Challenge:**
|
||||
- Different clients use different methodologies
|
||||
- Need consistent testing approach
|
||||
- Not always using BMad Method
|
||||
|
||||
**Model:** TEA Solo (bring to any client project)
|
||||
|
||||
**Workflow:**
|
||||
```
|
||||
Client project 1 (Scrum):
|
||||
- Import Jira stories
|
||||
- Run *test-design
|
||||
- Generate tests with *atdd/*automate
|
||||
- Deliver quality report with *test-review
|
||||
|
||||
Client project 2 (Kanban):
|
||||
- Import requirements from Notion
|
||||
- Same TEA workflow
|
||||
- Consistent quality across clients
|
||||
|
||||
Client project 3 (Ad-hoc):
|
||||
- Document requirements manually
|
||||
- Same TEA workflow
|
||||
- Same patterns, different context
|
||||
```
|
||||
|
||||
**Benefit:** Consistent testing approach regardless of client methodology.
|
||||
|
||||
## Choosing Your Model
|
||||
|
||||
### Start Here Questions
|
||||
|
||||
**Question 1:** Are you using BMad Method?
|
||||
- **No** → TEA Solo or TEA Lite or No TEA
|
||||
- **Yes** → TEA Integrated or No TEA
|
||||
|
||||
**Question 2:** Is this a new project?
|
||||
- **Yes** → TEA Integrated (Greenfield) or TEA Lite
|
||||
- **No** → TEA Integrated (Brownfield) or TEA Solo
|
||||
|
||||
**Question 3:** What's your testing maturity?
|
||||
- **Beginner** → TEA Lite
|
||||
- **Intermediate** → TEA Solo or Integrated
|
||||
- **Advanced** → TEA Integrated or No TEA (already expert)
|
||||
|
||||
**Question 4:** Do you need compliance/quality gates?
|
||||
- **Yes** → TEA Integrated (Enterprise)
|
||||
- **No** → Any model
|
||||
|
||||
**Question 5:** How much time can you invest?
|
||||
- **30 minutes** → TEA Lite
|
||||
- **Few hours** → TEA Solo
|
||||
- **Multiple days** → TEA Integrated
|
||||
|
||||
### Recommendation Matrix
|
||||
|
||||
| Your Context | Recommended Model | Alternative |
|
||||
|--------------|------------------|-------------|
|
||||
| BMad Method + new project | TEA Integrated (Greenfield) | TEA Lite (learning) |
|
||||
| BMad Method + existing code | TEA Integrated (Brownfield) | TEA Solo |
|
||||
| Non-BMad + need quality | TEA Solo | TEA Lite |
|
||||
| Just learning testing | TEA Lite | No TEA (learn basics first) |
|
||||
| Enterprise + compliance | TEA Integrated (Enterprise) | TEA Solo |
|
||||
| Established QA team | No TEA | TEA Solo (supplement) |
|
||||
|
||||
## Transitioning Between Models
|
||||
|
||||
### TEA Lite → TEA Solo
|
||||
|
||||
**When:** Outgrow beginner approach, need more workflows.
|
||||
|
||||
**Steps:**
|
||||
1. Continue using `*framework` and `*automate`
|
||||
2. Add `*test-design` for planning
|
||||
3. Add `*atdd` for TDD workflow
|
||||
4. Add `*test-review` for quality audits
|
||||
5. Add `*trace` for coverage tracking
|
||||
|
||||
**Timeline:** 2-4 weeks of gradual expansion
|
||||
|
||||
### TEA Solo → TEA Integrated
|
||||
|
||||
**When:** Adopt BMad Method, want full integration.
|
||||
|
||||
**Steps:**
|
||||
1. Install BMad Method (see installation guide)
|
||||
2. Run planning workflows (PRD, architecture)
|
||||
3. Integrate TEA into Phase 3 (system-level test design)
|
||||
4. Follow integrated lifecycle (per epic workflows)
|
||||
5. Add release gates (trace Phase 2)
|
||||
|
||||
**Timeline:** 1-2 sprints of transition
|
||||
|
||||
### TEA Integrated → TEA Solo
|
||||
|
||||
**When:** Moving away from BMad Method, keep TEA.
|
||||
|
||||
**Steps:**
|
||||
1. Export BMad artifacts (PRD, architecture, stories)
|
||||
2. Continue using TEA workflows standalone
|
||||
3. Skip BMad-specific integration
|
||||
4. Bring your own requirements to TEA
|
||||
|
||||
**Timeline:** Immediate (just skip BMad workflows)
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Pattern 1: TEA Lite for Learning, Then Choose
|
||||
|
||||
```
|
||||
Phase 1 (Week 1-2): TEA Lite
|
||||
- Learn with *automate on demo app
|
||||
- Understand TEA fundamentals
|
||||
- Low commitment
|
||||
|
||||
Phase 2 (Week 3-4): Evaluate
|
||||
- Try *test-design (planning)
|
||||
- Try *atdd (TDD)
|
||||
- See if value justifies investment
|
||||
|
||||
Phase 3 (Month 2+): Decide
|
||||
- Valuable → Expand to TEA Solo or Integrated
|
||||
- Not valuable → Stay with TEA Lite or No TEA
|
||||
```
|
||||
|
||||
### Pattern 2: TEA Solo for Quality, Skip Full Method
|
||||
|
||||
```
|
||||
Team decision:
|
||||
- Don't want full BMad Method (too heavyweight)
|
||||
- Want systematic testing (TEA benefits)
|
||||
|
||||
Approach: TEA Solo only
|
||||
- Use existing project management (Jira, Linear)
|
||||
- Use TEA for testing only
|
||||
- Get quality without methodology commitment
|
||||
```
|
||||
|
||||
### Pattern 3: Integrated for Critical, Lite for Non-Critical
|
||||
|
||||
```
|
||||
Critical features (payment, auth):
|
||||
- Full TEA Integrated workflow
|
||||
- Risk assessment, ATDD, quality gates
|
||||
- High confidence required
|
||||
|
||||
Non-critical features (UI tweaks):
|
||||
- TEA Lite or No TEA
|
||||
- Quick tests, minimal overhead
|
||||
- Move fast
|
||||
```
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
Each model uses different TEA workflows. See:
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Model details
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - Workflow reference
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Setup options
|
||||
|
||||
## Related Concepts
|
||||
|
||||
**Core TEA Concepts:**
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Risk assessment in different models
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Quality across all models
|
||||
- [Knowledge Base System](/docs/explanation/tea/knowledge-base-system.md) - Consistent patterns across models
|
||||
|
||||
**Technical Patterns:**
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Infrastructure in different models
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Reliability in all models
|
||||
|
||||
**Overview:**
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - 5 engagement models with cheat sheets
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - Design philosophy
|
||||
|
||||
## Practical Guides
|
||||
|
||||
**Getting Started:**
|
||||
- [TEA Lite Quickstart Tutorial](/docs/tutorials/getting-started/tea-lite-quickstart.md) - Model 3: TEA Lite
|
||||
|
||||
**Use-Case Guides:**
|
||||
- [Using TEA with Existing Tests](/docs/how-to/brownfield/use-tea-with-existing-tests.md) - Model 5: Brownfield
|
||||
- [Running TEA for Enterprise](/docs/how-to/brownfield/use-tea-for-enterprise.md) - Enterprise integration
|
||||
|
||||
**All Workflow Guides:**
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md) - Used in TEA Solo and Integrated
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md)
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md)
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md)
|
||||
- [How to Run Trace](/docs/how-to/workflows/run-trace.md)
|
||||
|
||||
## Reference
|
||||
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - All workflows explained
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Config per model
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - TEA Lite, TEA Solo, TEA Integrated terms
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,457 @@
|
|||
---
|
||||
title: "Fixture Architecture Explained"
|
||||
description: Understanding TEA's pure function → fixture → composition pattern for reusable test utilities
|
||||
---
|
||||
|
||||
# Fixture Architecture Explained
|
||||
|
||||
Fixture architecture is TEA's pattern for building reusable, testable, and composable test utilities. The core principle: build pure functions first, wrap in framework fixtures second.
|
||||
|
||||
## Overview
|
||||
|
||||
**The Pattern:**
|
||||
1. Write utility as pure function (unit-testable)
|
||||
2. Wrap in framework fixture (Playwright, Cypress)
|
||||
3. Compose fixtures with mergeTests (combine capabilities)
|
||||
4. Package for reuse across projects
|
||||
|
||||
**Why this order?**
|
||||
- Pure functions are easier to test
|
||||
- Fixtures depend on framework (less portable)
|
||||
- Composition happens at fixture level
|
||||
- Reusability maximized
|
||||
|
||||
### Fixture Architecture Flow
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'14px'}}}%%
|
||||
flowchart TD
|
||||
Start([Testing Need]) --> Pure[Step 1: Pure Function<br/>helpers/api-request.ts]
|
||||
Pure -->|Unit testable<br/>Framework agnostic| Fixture[Step 2: Fixture Wrapper<br/>fixtures/api-request.ts]
|
||||
Fixture -->|Injects framework<br/>dependencies| Compose[Step 3: Composition<br/>fixtures/index.ts]
|
||||
Compose -->|mergeTests| Use[Step 4: Use in Tests<br/>tests/**.spec.ts]
|
||||
|
||||
Pure -.->|Can test in isolation| UnitTest[Unit Tests<br/>No framework needed]
|
||||
Fixture -.->|Reusable pattern| Other[Other Projects<br/>Package export]
|
||||
Compose -.->|Combine utilities| Multi[Multiple Fixtures<br/>One test]
|
||||
|
||||
style Pure fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
|
||||
style Fixture fill:#fff3e0,stroke:#e65100,stroke-width:2px
|
||||
style Compose fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px
|
||||
style Use fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px
|
||||
style UnitTest fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px
|
||||
style Other fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px
|
||||
style Multi fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px
|
||||
```
|
||||
|
||||
**Benefits at Each Step:**
|
||||
1. **Pure Function:** Testable, portable, reusable
|
||||
2. **Fixture:** Framework integration, clean API
|
||||
3. **Composition:** Combine capabilities, flexible
|
||||
4. **Usage:** Simple imports, type-safe
|
||||
|
||||
## The Problem
|
||||
|
||||
### Framework-First Approach (Common Anti-Pattern)
|
||||
|
||||
```typescript
|
||||
// ❌ Bad: Built as fixture from the start
|
||||
export const test = base.extend({
|
||||
apiRequest: async ({ request }, use) => {
|
||||
await use(async (options) => {
|
||||
const response = await request.fetch(options.url, {
|
||||
method: options.method,
|
||||
data: options.data
|
||||
});
|
||||
|
||||
if (!response.ok()) {
|
||||
throw new Error(`API request failed: ${response.status()}`);
|
||||
}
|
||||
|
||||
return response.json();
|
||||
});
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Cannot unit test (requires Playwright context)
|
||||
- Tied to framework (not reusable in other tools)
|
||||
- Hard to compose with other fixtures
|
||||
- Difficult to mock for testing the utility itself
|
||||
|
||||
### Copy-Paste Utilities
|
||||
|
||||
```typescript
|
||||
// test-1.spec.ts
|
||||
test('test 1', async ({ request }) => {
|
||||
const response = await request.post('/api/users', { data: {...} });
|
||||
const body = await response.json();
|
||||
if (!response.ok()) throw new Error('Failed');
|
||||
// ... repeated in every test
|
||||
});
|
||||
|
||||
// test-2.spec.ts
|
||||
test('test 2', async ({ request }) => {
|
||||
const response = await request.post('/api/users', { data: {...} });
|
||||
const body = await response.json();
|
||||
if (!response.ok()) throw new Error('Failed');
|
||||
// ... same code repeated
|
||||
});
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Code duplication (violates DRY)
|
||||
- Inconsistent error handling
|
||||
- Hard to update (change 50 tests)
|
||||
- No shared behavior
|
||||
|
||||
## The Solution: Three-Step Pattern
|
||||
|
||||
### Step 1: Pure Function
|
||||
|
||||
```typescript
|
||||
// helpers/api-request.ts
|
||||
|
||||
/**
|
||||
* Make API request with automatic error handling
|
||||
* Pure function - no framework dependencies
|
||||
*/
|
||||
export async function apiRequest({
|
||||
request, // Passed in (dependency injection)
|
||||
method,
|
||||
url,
|
||||
data,
|
||||
headers = {}
|
||||
}: ApiRequestParams): Promise<ApiResponse> {
|
||||
const response = await request.fetch(url, {
|
||||
method,
|
||||
data,
|
||||
headers
|
||||
});
|
||||
|
||||
if (!response.ok()) {
|
||||
throw new Error(`API request failed: ${response.status()}`);
|
||||
}
|
||||
|
||||
return {
|
||||
status: response.status(),
|
||||
body: await response.json()
|
||||
};
|
||||
}
|
||||
|
||||
// ✅ Can unit test this function!
|
||||
describe('apiRequest', () => {
|
||||
it('should throw on non-OK response', async () => {
|
||||
const mockRequest = {
|
||||
fetch: vi.fn().mockResolvedValue({ ok: () => false, status: () => 500 })
|
||||
};
|
||||
|
||||
await expect(apiRequest({
|
||||
request: mockRequest,
|
||||
method: 'GET',
|
||||
url: '/api/test'
|
||||
})).rejects.toThrow('API request failed: 500');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Unit testable (mock dependencies)
|
||||
- Framework-agnostic (works with any HTTP client)
|
||||
- Easy to reason about (pure function)
|
||||
- Portable (can use in Node scripts, CLI tools)
|
||||
|
||||
### Step 2: Fixture Wrapper
|
||||
|
||||
```typescript
|
||||
// fixtures/api-request.ts
|
||||
import { test as base } from '@playwright/test';
|
||||
import { apiRequest as apiRequestFn } from '../helpers/api-request';
|
||||
|
||||
/**
|
||||
* Playwright fixture wrapping the pure function
|
||||
*/
|
||||
export const test = base.extend<{ apiRequest: typeof apiRequestFn }>({
|
||||
apiRequest: async ({ request }, use) => {
|
||||
// Inject framework dependency (request)
|
||||
await use((params) => apiRequestFn({ request, ...params }));
|
||||
}
|
||||
});
|
||||
|
||||
export { expect } from '@playwright/test';
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Fixture provides framework context (request)
|
||||
- Pure function handles logic
|
||||
- Clean separation of concerns
|
||||
- Can swap frameworks (Cypress, etc.) by changing wrapper only
|
||||
|
||||
### Step 3: Composition with mergeTests
|
||||
|
||||
```typescript
|
||||
// fixtures/index.ts
|
||||
import { mergeTests } from '@playwright/test';
|
||||
import { test as apiRequestTest } from './api-request';
|
||||
import { test as authSessionTest } from './auth-session';
|
||||
import { test as logTest } from './log';
|
||||
|
||||
/**
|
||||
* Compose all fixtures into one test
|
||||
*/
|
||||
export const test = mergeTests(
|
||||
apiRequestTest,
|
||||
authSessionTest,
|
||||
logTest
|
||||
);
|
||||
|
||||
export { expect } from '@playwright/test';
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
// tests/profile.spec.ts
|
||||
import { test, expect } from '../support/fixtures';
|
||||
|
||||
test('should update profile', async ({ apiRequest, authToken, log }) => {
|
||||
log.info('Starting profile update test');
|
||||
|
||||
// Use API request fixture (matches pure function signature)
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'PATCH',
|
||||
url: '/api/profile',
|
||||
data: { name: 'New Name' },
|
||||
headers: { Authorization: `Bearer ${authToken}` }
|
||||
});
|
||||
|
||||
expect(status).toBe(200);
|
||||
expect(body.name).toBe('New Name');
|
||||
|
||||
log.info('Profile updated successfully');
|
||||
});
|
||||
```
|
||||
|
||||
**Note:** This example uses the vanilla pure function signature (`url`, `data`). Playwright Utils uses different parameter names (`path`, `body`). See [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) for the utilities API.
|
||||
|
||||
**Note:** `authToken` requires auth-session fixture setup with provider configuration. See [auth-session documentation](https://seontechnologies.github.io/playwright-utils/auth-session.html).
|
||||
|
||||
**Benefits:**
|
||||
- Use multiple fixtures in one test
|
||||
- No manual composition needed
|
||||
- Type-safe (TypeScript knows all fixture types)
|
||||
- Clean imports
|
||||
|
||||
## How It Works in TEA
|
||||
|
||||
### TEA Generates This Pattern
|
||||
|
||||
When you run `*framework` with `tea_use_playwright_utils: true`:
|
||||
|
||||
**TEA scaffolds:**
|
||||
```
|
||||
tests/
|
||||
├── support/
|
||||
│ ├── helpers/ # Pure functions
|
||||
│ │ ├── api-request.ts
|
||||
│ │ └── auth-session.ts
|
||||
│ └── fixtures/ # Framework wrappers
|
||||
│ ├── api-request.ts
|
||||
│ ├── auth-session.ts
|
||||
│ └── index.ts # Composition
|
||||
└── e2e/
|
||||
└── example.spec.ts # Uses composed fixtures
|
||||
```
|
||||
|
||||
### TEA Reviews Against This Pattern
|
||||
|
||||
When you run `*test-review`:
|
||||
|
||||
**TEA checks:**
|
||||
- Are utilities pure functions? ✓
|
||||
- Are fixtures minimal wrappers? ✓
|
||||
- Is composition used? ✓
|
||||
- Can utilities be unit tested? ✓
|
||||
|
||||
## Package Export Pattern
|
||||
|
||||
### Make Fixtures Reusable Across Projects
|
||||
|
||||
**Option 1: Build Your Own (Vanilla)**
|
||||
```json
|
||||
// package.json
|
||||
{
|
||||
"name": "@company/test-utils",
|
||||
"exports": {
|
||||
"./api-request": "./fixtures/api-request.ts",
|
||||
"./auth-session": "./fixtures/auth-session.ts",
|
||||
"./log": "./fixtures/log.ts"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { test as apiTest } from '@company/test-utils/api-request';
|
||||
import { test as authTest } from '@company/test-utils/auth-session';
|
||||
import { mergeTests } from '@playwright/test';
|
||||
|
||||
export const test = mergeTests(apiTest, authTest);
|
||||
```
|
||||
|
||||
**Option 2: Use Playwright Utils (Recommended)**
|
||||
```bash
|
||||
npm install -D @seontechnologies/playwright-utils
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { test as base } from '@playwright/test';
|
||||
import { mergeTests } from '@playwright/test';
|
||||
import { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { createAuthFixtures } from '@seontechnologies/playwright-utils/auth-session';
|
||||
|
||||
const authFixtureTest = base.extend(createAuthFixtures());
|
||||
export const test = mergeTests(apiRequestFixture, authFixtureTest);
|
||||
// Production-ready utilities, battle-tested!
|
||||
```
|
||||
|
||||
**Note:** Auth-session requires provider configuration. See [auth-session setup guide](https://seontechnologies.github.io/playwright-utils/auth-session.html).
|
||||
|
||||
**Why Playwright Utils:**
|
||||
- Already built, tested, and maintained
|
||||
- Consistent patterns across projects
|
||||
- 11 utilities available (API, auth, network, logging, files)
|
||||
- Community support and documentation
|
||||
- Regular updates and improvements
|
||||
|
||||
**When to Build Your Own:**
|
||||
- Company-specific patterns
|
||||
- Custom authentication systems
|
||||
- Unique requirements not covered by utilities
|
||||
|
||||
## Comparison: Good vs Bad Patterns
|
||||
|
||||
### Anti-Pattern: God Fixture
|
||||
|
||||
```typescript
|
||||
// ❌ Bad: Everything in one fixture
|
||||
export const test = base.extend({
|
||||
testUtils: async ({ page, request, context }, use) => {
|
||||
await use({
|
||||
// 50 different methods crammed into one fixture
|
||||
apiRequest: async (...) => { },
|
||||
login: async (...) => { },
|
||||
createUser: async (...) => { },
|
||||
deleteUser: async (...) => { },
|
||||
uploadFile: async (...) => { },
|
||||
// ... 45 more methods
|
||||
});
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Cannot test individual utilities
|
||||
- Cannot compose (all-or-nothing)
|
||||
- Cannot reuse specific utilities
|
||||
- Hard to maintain (1000+ line file)
|
||||
|
||||
### Good Pattern: Single-Concern Fixtures
|
||||
|
||||
```typescript
|
||||
// ✅ Good: One concern per fixture
|
||||
|
||||
// api-request.ts
|
||||
export const test = base.extend({ apiRequest });
|
||||
|
||||
// auth-session.ts
|
||||
export const test = base.extend({ authSession });
|
||||
|
||||
// log.ts
|
||||
export const test = base.extend({ log });
|
||||
|
||||
// Compose as needed
|
||||
import { mergeTests } from '@playwright/test';
|
||||
export const test = mergeTests(apiRequestTest, authSessionTest, logTest);
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Each fixture is unit-testable
|
||||
- Compose only what you need
|
||||
- Reuse individual fixtures
|
||||
- Easy to maintain (small files)
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
For detailed fixture architecture patterns, see the knowledge base:
|
||||
- [Knowledge Base Index - Architecture & Fixtures](/docs/reference/tea/knowledge-base.md)
|
||||
- [Complete Knowledge Base Index](/docs/reference/tea/knowledge-base.md)
|
||||
|
||||
## When to Use This Pattern
|
||||
|
||||
### Always Use For:
|
||||
|
||||
**Reusable utilities:**
|
||||
- API request helpers
|
||||
- Authentication handlers
|
||||
- File operations
|
||||
- Network mocking
|
||||
|
||||
**Test infrastructure:**
|
||||
- Shared fixtures across teams
|
||||
- Packaged utilities (playwright-utils)
|
||||
- Company-wide test standards
|
||||
|
||||
### Consider Skipping For:
|
||||
|
||||
**One-off test setup:**
|
||||
```typescript
|
||||
// Simple one-time setup - inline is fine
|
||||
test.beforeEach(async ({ page }) => {
|
||||
await page.goto('/');
|
||||
await page.click('#accept-cookies');
|
||||
});
|
||||
```
|
||||
|
||||
**Test-specific helpers:**
|
||||
```typescript
|
||||
// Used in one test file only - keep local
|
||||
function createTestUser(name: string) {
|
||||
return { name, email: `${name}@test.com` };
|
||||
}
|
||||
```
|
||||
|
||||
## Related Concepts
|
||||
|
||||
**Core TEA Concepts:**
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Quality standards fixtures enforce
|
||||
- [Knowledge Base System](/docs/explanation/tea/knowledge-base-system.md) - Fixture patterns in knowledge base
|
||||
|
||||
**Technical Patterns:**
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Network fixtures explained
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Fixture complexity matches risk
|
||||
|
||||
**Overview:**
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Fixture architecture in workflows
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - Why fixtures matter
|
||||
|
||||
## Practical Guides
|
||||
|
||||
**Setup Guides:**
|
||||
- [How to Set Up Test Framework](/docs/how-to/workflows/setup-test-framework.md) - TEA scaffolds fixtures
|
||||
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) - Production-ready fixtures
|
||||
|
||||
**Workflow Guides:**
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Using fixtures in tests
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Fixture composition examples
|
||||
|
||||
## Reference
|
||||
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - *framework command
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Fixture architecture fragments
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - Fixture architecture term
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,554 @@
|
|||
---
|
||||
title: "Knowledge Base System Explained"
|
||||
description: Understanding how TEA uses tea-index.csv for context engineering and consistent test quality
|
||||
---
|
||||
|
||||
# Knowledge Base System Explained
|
||||
|
||||
TEA's knowledge base system is how context engineering works - automatically loading domain-specific standards into AI context so tests are consistently high-quality regardless of prompt variation.
|
||||
|
||||
## Overview
|
||||
|
||||
**The Problem:** AI without context produces inconsistent results.
|
||||
|
||||
**Traditional approach:**
|
||||
```
|
||||
User: "Write tests for login"
|
||||
AI: [Generates tests with random quality]
|
||||
- Sometimes uses hard waits
|
||||
- Sometimes uses good patterns
|
||||
- Inconsistent across sessions
|
||||
- Quality depends on prompt
|
||||
```
|
||||
|
||||
**TEA with knowledge base:**
|
||||
```
|
||||
User: "Write tests for login"
|
||||
TEA: [Loads test-quality.md, network-first.md, auth-session.md]
|
||||
TEA: [Generates tests following established patterns]
|
||||
- Always uses network-first patterns
|
||||
- Always uses proper fixtures
|
||||
- Consistent across all sessions
|
||||
- Quality independent of prompt
|
||||
```
|
||||
|
||||
**Result:** Systematic quality, not random chance.
|
||||
|
||||
## The Problem
|
||||
|
||||
### Prompt-Driven Testing = Inconsistency
|
||||
|
||||
**Session 1:**
|
||||
```
|
||||
User: "Write tests for profile editing"
|
||||
|
||||
AI: [No context loaded]
|
||||
// Generates test with hard waits
|
||||
await page.waitForTimeout(3000);
|
||||
```
|
||||
|
||||
**Session 2:**
|
||||
```
|
||||
User: "Write comprehensive tests for profile editing with best practices"
|
||||
|
||||
AI: [Still no systematic context]
|
||||
// Generates test with some improvements, but still issues
|
||||
await page.waitForSelector('.success', { timeout: 10000 });
|
||||
```
|
||||
|
||||
**Session 3:**
|
||||
```
|
||||
User: "Write tests using network-first patterns and proper fixtures"
|
||||
|
||||
AI: [Better prompt, but still reinventing patterns]
|
||||
// Generates test with network-first, but inconsistent with other tests
|
||||
```
|
||||
|
||||
**Problem:** Quality depends on prompt engineering skill, no consistency.
|
||||
|
||||
### Knowledge Drift
|
||||
|
||||
Without a knowledge base:
|
||||
- Team A uses pattern X
|
||||
- Team B uses pattern Y
|
||||
- Both work, but inconsistent
|
||||
- No single source of truth
|
||||
- Patterns drift over time
|
||||
|
||||
## The Solution: tea-index.csv Manifest
|
||||
|
||||
### How It Works
|
||||
|
||||
**1. Manifest Defines Fragments**
|
||||
|
||||
`src/modules/bmm/testarch/tea-index.csv`:
|
||||
```csv
|
||||
id,name,description,tags,fragment_file
|
||||
test-quality,Test Quality,Execution limits and isolation rules,quality;standards,knowledge/test-quality.md
|
||||
network-first,Network-First Safeguards,Intercept-before-navigate workflow,network;stability,knowledge/network-first.md
|
||||
fixture-architecture,Fixture Architecture,Composable fixture patterns,fixtures;architecture,knowledge/fixture-architecture.md
|
||||
```
|
||||
|
||||
**2. Workflow Loads Relevant Fragments**
|
||||
|
||||
When user runs `*atdd`:
|
||||
```
|
||||
TEA reads tea-index.csv
|
||||
Identifies fragments needed for ATDD:
|
||||
- test-quality.md (quality standards)
|
||||
- network-first.md (avoid flakiness)
|
||||
- component-tdd.md (TDD patterns)
|
||||
- fixture-architecture.md (reusable fixtures)
|
||||
- data-factories.md (test data)
|
||||
|
||||
Loads only these 5 fragments (not all 33)
|
||||
Generates tests following these patterns
|
||||
```
|
||||
|
||||
**3. Consistent Output**
|
||||
|
||||
Every time `*atdd` runs:
|
||||
- Same fragments loaded
|
||||
- Same patterns applied
|
||||
- Same quality standards
|
||||
- Consistent test structure
|
||||
|
||||
**Result:** Tests look like they were written by the same expert, every time.
|
||||
|
||||
### Knowledge Base Loading Diagram
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'14px'}}}%%
|
||||
flowchart TD
|
||||
User([User: *atdd]) --> Workflow[TEA Workflow<br/>Triggered]
|
||||
Workflow --> Read[Read Manifest<br/>tea-index.csv]
|
||||
|
||||
Read --> Identify{Identify Relevant<br/>Fragments for ATDD}
|
||||
|
||||
Identify -->|Needed| L1[✓ test-quality.md]
|
||||
Identify -->|Needed| L2[✓ network-first.md]
|
||||
Identify -->|Needed| L3[✓ component-tdd.md]
|
||||
Identify -->|Needed| L4[✓ data-factories.md]
|
||||
Identify -->|Needed| L5[✓ fixture-architecture.md]
|
||||
|
||||
Identify -.->|Skip| S1[✗ contract-testing.md]
|
||||
Identify -.->|Skip| S2[✗ burn-in.md]
|
||||
Identify -.->|Skip| S3[+ 26 other fragments]
|
||||
|
||||
L1 --> Context[AI Context<br/>5 fragments loaded]
|
||||
L2 --> Context
|
||||
L3 --> Context
|
||||
L4 --> Context
|
||||
L5 --> Context
|
||||
|
||||
Context --> Gen[Generate Tests<br/>Following patterns]
|
||||
Gen --> Out([Consistent Output<br/>Same quality every time])
|
||||
|
||||
style User fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
|
||||
style Read fill:#fff3e0,stroke:#e65100,stroke-width:2px
|
||||
style L1 fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
|
||||
style L2 fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
|
||||
style L3 fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
|
||||
style L4 fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
|
||||
style L5 fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
|
||||
style S1 fill:#e0e0e0,stroke:#616161,stroke-width:1px
|
||||
style S2 fill:#e0e0e0,stroke:#616161,stroke-width:1px
|
||||
style S3 fill:#e0e0e0,stroke:#616161,stroke-width:1px
|
||||
style Context fill:#f3e5f5,stroke:#6a1b9a,stroke-width:3px
|
||||
style Out fill:#4caf50,stroke:#1b5e20,stroke-width:3px,color:#fff
|
||||
```
|
||||
|
||||
## Fragment Structure
|
||||
|
||||
### Anatomy of a Fragment
|
||||
|
||||
Each fragment follows this structure:
|
||||
|
||||
```markdown
|
||||
# Fragment Name
|
||||
|
||||
## Principle
|
||||
[One sentence - what is this pattern?]
|
||||
|
||||
## Rationale
|
||||
[Why use this instead of alternatives?]
|
||||
Why this pattern exists
|
||||
Problems it solves
|
||||
Benefits it provides
|
||||
|
||||
## Pattern Examples
|
||||
|
||||
### Example 1: Basic Usage
|
||||
```code
|
||||
[Runnable code example]
|
||||
```
|
||||
[Explanation of example]
|
||||
|
||||
### Example 2: Advanced Pattern
|
||||
```code
|
||||
[More complex example]
|
||||
```
|
||||
[Explanation]
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
### Don't Do This
|
||||
```code
|
||||
[Bad code example]
|
||||
```
|
||||
[Why it's bad]
|
||||
[What breaks]
|
||||
|
||||
## Related Patterns
|
||||
- [Link to related fragment]
|
||||
```
|
||||
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
### Example: test-quality.md Fragment
|
||||
|
||||
```markdown
|
||||
# Test Quality
|
||||
|
||||
## Principle
|
||||
Tests must be deterministic, isolated, explicit, focused, and fast.
|
||||
|
||||
## Rationale
|
||||
Tests that fail randomly, depend on each other, or take too long lose team trust.
|
||||
[... detailed explanation ...]
|
||||
|
||||
## Pattern Examples
|
||||
|
||||
### Example 1: Deterministic Test
|
||||
```typescript
|
||||
// ✅ Wait for actual response, not timeout
|
||||
const promise = page.waitForResponse(matcher);
|
||||
await page.click('button');
|
||||
await promise;
|
||||
```
|
||||
|
||||
### Example 2: Isolated Test
|
||||
```typescript
|
||||
// ✅ Self-cleaning test
|
||||
test('test', async ({ page }) => {
|
||||
const userId = await createTestUser();
|
||||
// ... test logic ...
|
||||
await deleteTestUser(userId); // Cleanup
|
||||
});
|
||||
```
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
### Hard Waits
|
||||
```typescript
|
||||
// ❌ Non-deterministic
|
||||
await page.waitForTimeout(3000);
|
||||
```
|
||||
[Why this causes flakiness]
|
||||
```
|
||||
|
||||
**Total:** 24.5 KB, 12 code examples
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
|
||||
## How TEA Uses the Knowledge Base
|
||||
|
||||
### Workflow-Specific Loading
|
||||
|
||||
**Different workflows load different fragments:**
|
||||
|
||||
| Workflow | Fragments Loaded | Purpose |
|
||||
|----------|-----------------|---------|
|
||||
| `*framework` | fixture-architecture, playwright-config, fixtures-composition | Infrastructure patterns |
|
||||
| `*test-design` | test-quality, test-priorities-matrix, risk-governance | Planning standards |
|
||||
| `*atdd` | test-quality, component-tdd, network-first, data-factories | TDD patterns |
|
||||
| `*automate` | test-quality, test-levels-framework, selector-resilience | Comprehensive generation |
|
||||
| `*test-review` | All quality/resilience/debugging fragments | Full audit patterns |
|
||||
| `*ci` | ci-burn-in, burn-in, selective-testing | CI/CD optimization |
|
||||
|
||||
**Benefit:** Only load what's needed (focused context, no bloat).
|
||||
|
||||
### Dynamic Fragment Selection
|
||||
|
||||
TEA doesn't load all 33 fragments at once:
|
||||
|
||||
```
|
||||
User runs: *atdd for authentication feature
|
||||
|
||||
TEA analyzes context:
|
||||
- Feature type: Authentication
|
||||
- Relevant fragments:
|
||||
- test-quality.md (always loaded)
|
||||
- auth-session.md (auth patterns)
|
||||
- network-first.md (avoid flakiness)
|
||||
- email-auth.md (if email-based auth)
|
||||
- data-factories.md (test users)
|
||||
|
||||
Skips:
|
||||
- contract-testing.md (not relevant)
|
||||
- feature-flags.md (not relevant)
|
||||
- file-utils.md (not relevant)
|
||||
|
||||
Result: 5 relevant fragments loaded, 28 skipped
|
||||
```
|
||||
|
||||
**Benefit:** Focused context = better results, lower token usage.
|
||||
|
||||
## Context Engineering in Practice
|
||||
|
||||
### Example: Consistent Test Generation
|
||||
|
||||
**Without Knowledge Base (Vanilla Playwright, Random Quality):**
|
||||
```
|
||||
Session 1: User runs *atdd
|
||||
AI: [Guesses patterns from general knowledge]
|
||||
|
||||
Generated:
|
||||
test('api test', async ({ request }) => {
|
||||
const response = await request.get('/api/users');
|
||||
await page.waitForTimeout(2000); // Hard wait
|
||||
const users = await response.json();
|
||||
// Random quality
|
||||
});
|
||||
|
||||
Session 2: User runs *atdd (different day)
|
||||
AI: [Different random patterns]
|
||||
|
||||
Generated:
|
||||
test('api test', async ({ request }) => {
|
||||
const response = await request.get('/api/users');
|
||||
const users = await response.json();
|
||||
// Better but inconsistent
|
||||
});
|
||||
|
||||
Result: Inconsistent quality, random patterns
|
||||
```
|
||||
|
||||
**With Knowledge Base (TEA + Playwright Utils):**
|
||||
```
|
||||
Session 1: User runs *atdd
|
||||
TEA: [Loads test-quality.md, network-first.md, api-request.md from tea-index.csv]
|
||||
|
||||
Generated:
|
||||
import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
|
||||
test('should fetch users', async ({ apiRequest }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/users'
|
||||
}).validateSchema(UsersSchema); // Chained validation
|
||||
|
||||
expect(status).toBe(200);
|
||||
expect(body).toBeInstanceOf(Array);
|
||||
});
|
||||
|
||||
Session 2: User runs *atdd (different day)
|
||||
TEA: [Loads same fragments from tea-index.csv]
|
||||
|
||||
Generated: Identical pattern, same quality
|
||||
|
||||
Result: Systematic quality, established patterns (ALWAYS uses apiRequest utility when playwright-utils enabled)
|
||||
```
|
||||
|
||||
**Key Difference:**
|
||||
- **Without KB:** Random patterns, inconsistent APIs
|
||||
- **With KB:** Always uses `apiRequest` utility, always validates schemas, always returns `{ status, body }`
|
||||
|
||||
### Example: Test Review Consistency
|
||||
|
||||
**Without Knowledge Base:**
|
||||
```
|
||||
*test-review session 1:
|
||||
"This test looks okay" [50 issues missed]
|
||||
|
||||
*test-review session 2:
|
||||
"This test has some issues" [Different issues flagged]
|
||||
|
||||
Result: Inconsistent feedback
|
||||
```
|
||||
|
||||
**With Knowledge Base:**
|
||||
```
|
||||
*test-review session 1:
|
||||
[Loads all quality fragments]
|
||||
Flags: 12 hard waits, 5 conditionals (based on test-quality.md)
|
||||
|
||||
*test-review session 2:
|
||||
[Loads same fragments]
|
||||
Flags: Same issues with same explanations
|
||||
|
||||
Result: Consistent, reliable feedback
|
||||
```
|
||||
|
||||
## Maintaining the Knowledge Base
|
||||
|
||||
### When to Add a Fragment
|
||||
|
||||
**Good reasons:**
|
||||
- Pattern is used across multiple workflows
|
||||
- Standard is non-obvious (needs documentation)
|
||||
- Team asks "how should we handle X?" repeatedly
|
||||
- New tool integration (e.g., new testing library)
|
||||
|
||||
**Bad reasons:**
|
||||
- One-off pattern (document in test file instead)
|
||||
- Obvious pattern (everyone knows this)
|
||||
- Experimental (not proven yet)
|
||||
|
||||
### Fragment Quality Standards
|
||||
|
||||
**Good fragment:**
|
||||
- Principle stated in one sentence
|
||||
- Rationale explains why clearly
|
||||
- 3+ pattern examples with code
|
||||
- Anti-patterns shown (what not to do)
|
||||
- Self-contained (minimal dependencies)
|
||||
|
||||
**Example size:** 10-30 KB optimal
|
||||
|
||||
### Updating Existing Fragments
|
||||
|
||||
**When to update:**
|
||||
- Pattern evolved (better approach discovered)
|
||||
- Tool updated (new Playwright API)
|
||||
- Team feedback (pattern unclear)
|
||||
- Bug in example code
|
||||
|
||||
**How to update:**
|
||||
1. Edit fragment markdown file
|
||||
2. Update examples
|
||||
3. Test with affected workflows
|
||||
4. Ensure no breaking changes
|
||||
|
||||
**No need to update tea-index.csv** unless description/tags change.
|
||||
|
||||
## Benefits of Knowledge Base System
|
||||
|
||||
### 1. Consistency
|
||||
|
||||
**Before:** Test quality varies by who wrote it
|
||||
**After:** All tests follow same patterns (TEA-generated or reviewed)
|
||||
|
||||
### 2. Onboarding
|
||||
|
||||
**Before:** New team member reads 20 documents, asks 50 questions
|
||||
**After:** New team member runs `*atdd`, sees patterns in generated code, learns by example
|
||||
|
||||
### 3. Quality Gates
|
||||
|
||||
**Before:** "Is this test good?" → subjective opinion
|
||||
**After:** "*test-review" → objective score against knowledge base
|
||||
|
||||
### 4. Pattern Evolution
|
||||
|
||||
**Before:** Update tests manually across 100 files
|
||||
**After:** Update fragment once, all new tests use new pattern
|
||||
|
||||
### 5. Cross-Project Reuse
|
||||
|
||||
**Before:** Reinvent patterns for each project
|
||||
**After:** Same fragments across all BMad projects (consistency at scale)
|
||||
|
||||
## Comparison: With vs Without Knowledge Base
|
||||
|
||||
### Scenario: Testing Async Background Job
|
||||
|
||||
**Without Knowledge Base:**
|
||||
|
||||
Developer 1:
|
||||
```typescript
|
||||
// Uses hard wait
|
||||
await page.click('button');
|
||||
await page.waitForTimeout(10000); // Hope job finishes
|
||||
```
|
||||
|
||||
Developer 2:
|
||||
```typescript
|
||||
// Uses polling
|
||||
await page.click('button');
|
||||
for (let i = 0; i < 10; i++) {
|
||||
const status = await page.locator('.status').textContent();
|
||||
if (status === 'complete') break;
|
||||
await page.waitForTimeout(1000);
|
||||
}
|
||||
```
|
||||
|
||||
Developer 3:
|
||||
```typescript
|
||||
// Uses waitForSelector
|
||||
await page.click('button');
|
||||
await page.waitForSelector('.success', { timeout: 30000 });
|
||||
```
|
||||
|
||||
**Result:** 3 different patterns, all suboptimal.
|
||||
|
||||
**With Knowledge Base (recurse.md fragment):**
|
||||
|
||||
All developers:
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
|
||||
test('job completion', async ({ apiRequest, recurse }) => {
|
||||
// Start async job
|
||||
const { body: job } = await apiRequest({
|
||||
method: 'POST',
|
||||
path: '/api/jobs'
|
||||
});
|
||||
|
||||
// Poll until complete (correct API: command, predicate, options)
|
||||
const result = await recurse(
|
||||
() => apiRequest({ method: 'GET', path: `/api/jobs/${job.id}` }),
|
||||
(response) => response.body.status === 'completed', // response.body from apiRequest
|
||||
{
|
||||
timeout: 30000,
|
||||
interval: 2000,
|
||||
log: 'Waiting for job to complete'
|
||||
}
|
||||
);
|
||||
|
||||
expect(result.body.status).toBe('completed');
|
||||
});
|
||||
```
|
||||
|
||||
**Result:** Consistent pattern using correct playwright-utils API (command, predicate, options).
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
For details on the knowledge base index, see:
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md)
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md)
|
||||
|
||||
## Related Concepts
|
||||
|
||||
**Core TEA Concepts:**
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Standards in knowledge base
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Risk patterns in knowledge base
|
||||
- [Engagement Models](/docs/explanation/tea/engagement-models.md) - Knowledge base across all models
|
||||
|
||||
**Technical Patterns:**
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Fixture patterns in knowledge base
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Network patterns in knowledge base
|
||||
|
||||
**Overview:**
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Knowledge base in workflows
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - **Foundation: Context engineering philosophy** (why knowledge base solves AI test problems)
|
||||
|
||||
## Practical Guides
|
||||
|
||||
**All Workflow Guides Use Knowledge Base:**
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md)
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md)
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md)
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md)
|
||||
|
||||
**Integration:**
|
||||
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) - PW-Utils in knowledge base
|
||||
|
||||
## Reference
|
||||
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Complete fragment index
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - Which workflows load which fragments
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Config affects fragment loading
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - Context engineering, knowledge fragment terms
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,853 @@
|
|||
---
|
||||
title: "Network-First Patterns Explained"
|
||||
description: Understanding how TEA eliminates test flakiness by waiting for actual network responses
|
||||
---
|
||||
|
||||
# Network-First Patterns Explained
|
||||
|
||||
Network-first patterns are TEA's solution to test flakiness. Instead of guessing how long to wait with fixed timeouts, wait for the actual network event that causes UI changes.
|
||||
|
||||
## Overview
|
||||
|
||||
**The Core Principle:**
|
||||
UI changes because APIs respond. Wait for the API response, not an arbitrary timeout.
|
||||
|
||||
**Traditional approach:**
|
||||
```typescript
|
||||
await page.click('button');
|
||||
await page.waitForTimeout(3000); // Hope 3 seconds is enough
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
```
|
||||
|
||||
**Network-first approach:**
|
||||
```typescript
|
||||
const responsePromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/submit') && resp.ok()
|
||||
);
|
||||
await page.click('button');
|
||||
await responsePromise; // Wait for actual response
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
```
|
||||
|
||||
**Result:** Deterministic tests that wait exactly as long as needed.
|
||||
|
||||
## The Problem
|
||||
|
||||
### Hard Waits Create Flakiness
|
||||
|
||||
```typescript
|
||||
// ❌ The flaky test pattern
|
||||
test('should submit form', async ({ page }) => {
|
||||
await page.fill('#name', 'Test User');
|
||||
await page.click('button[type="submit"]');
|
||||
|
||||
await page.waitForTimeout(2000); // Wait 2 seconds
|
||||
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Why this fails:**
|
||||
- **Fast network:** Wastes 1.5 seconds waiting
|
||||
- **Slow network:** Not enough time, test fails
|
||||
- **CI environment:** Slower than local, fails randomly
|
||||
- **Under load:** API takes 3 seconds, test fails
|
||||
|
||||
**Result:** "Works on my machine" syndrome, flaky CI.
|
||||
|
||||
### The Timeout Escalation Trap
|
||||
|
||||
```typescript
|
||||
// Developer sees flaky test
|
||||
await page.waitForTimeout(2000); // Failed in CI
|
||||
|
||||
// Increases timeout
|
||||
await page.waitForTimeout(5000); // Still fails sometimes
|
||||
|
||||
// Increases again
|
||||
await page.waitForTimeout(10000); // Now it passes... slowly
|
||||
|
||||
// Problem: Now EVERY test waits 10 seconds
|
||||
// Suite that took 5 minutes now takes 30 minutes
|
||||
```
|
||||
|
||||
**Result:** Slow, still-flaky tests.
|
||||
|
||||
### Race Conditions
|
||||
|
||||
```typescript
|
||||
// ❌ Navigate-then-wait race condition
|
||||
test('should load dashboard data', async ({ page }) => {
|
||||
await page.goto('/dashboard'); // Navigation starts
|
||||
|
||||
// Race condition! API might not have responded yet
|
||||
await expect(page.locator('.data-table')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. `goto()` starts navigation
|
||||
2. Page loads HTML
|
||||
3. JavaScript requests `/api/dashboard`
|
||||
4. Test checks for `.data-table` BEFORE API responds
|
||||
5. Test fails intermittently
|
||||
|
||||
**Result:** "Sometimes it works, sometimes it doesn't."
|
||||
|
||||
## The Solution: Intercept-Before-Navigate
|
||||
|
||||
### Wait for Response Before Asserting
|
||||
|
||||
```typescript
|
||||
// ✅ Good: Network-first pattern
|
||||
test('should load dashboard data', async ({ page }) => {
|
||||
// Set up promise BEFORE navigation
|
||||
const dashboardPromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/dashboard') && resp.ok()
|
||||
);
|
||||
|
||||
// Navigate
|
||||
await page.goto('/dashboard');
|
||||
|
||||
// Wait for API response
|
||||
const response = await dashboardPromise;
|
||||
const data = await response.json();
|
||||
|
||||
// Now assert UI
|
||||
await expect(page.locator('.data-table')).toBeVisible();
|
||||
await expect(page.locator('.data-table tr')).toHaveCount(data.items.length);
|
||||
});
|
||||
```
|
||||
|
||||
**Why this works:**
|
||||
- Wait set up BEFORE navigation (no race)
|
||||
- Wait for actual API response (deterministic)
|
||||
- No fixed timeout (fast when API is fast)
|
||||
- Validates API response (catch backend errors)
|
||||
|
||||
**With Playwright Utils (Even Cleaner):**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
|
||||
test('should load dashboard data', async ({ page, interceptNetworkCall }) => {
|
||||
// Set up interception BEFORE navigation
|
||||
const dashboardCall = interceptNetworkCall({
|
||||
method: 'GET',
|
||||
url: '**/api/dashboard'
|
||||
});
|
||||
|
||||
// Navigate
|
||||
await page.goto('/dashboard');
|
||||
|
||||
// Wait for API response (automatic JSON parsing)
|
||||
const { status, responseJson: data } = await dashboardCall;
|
||||
|
||||
// Validate API response
|
||||
expect(status).toBe(200);
|
||||
expect(data.items).toBeDefined();
|
||||
|
||||
// Assert UI matches API data
|
||||
await expect(page.locator('.data-table')).toBeVisible();
|
||||
await expect(page.locator('.data-table tr')).toHaveCount(data.items.length);
|
||||
});
|
||||
```
|
||||
|
||||
**Playwright Utils Benefits:**
|
||||
- Automatic JSON parsing (no `await response.json()`)
|
||||
- Returns `{ status, responseJson, requestJson }` structure
|
||||
- Cleaner API (no need to check `resp.ok()`)
|
||||
- Same intercept-before-navigate pattern
|
||||
|
||||
### Intercept-Before-Navigate Pattern
|
||||
|
||||
**Key insight:** Set up wait BEFORE triggering the action.
|
||||
|
||||
```typescript
|
||||
// ✅ Pattern: Intercept → Action → Await
|
||||
|
||||
// 1. Intercept (set up wait)
|
||||
const promise = page.waitForResponse(matcher);
|
||||
|
||||
// 2. Action (trigger request)
|
||||
await page.click('button');
|
||||
|
||||
// 3. Await (wait for actual response)
|
||||
await promise;
|
||||
```
|
||||
|
||||
**Why this order:**
|
||||
- `waitForResponse()` starts listening immediately
|
||||
- Then trigger the action that makes the request
|
||||
- Then wait for the promise to resolve
|
||||
- No race condition possible
|
||||
|
||||
#### Intercept-Before-Navigate Flow
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'14px'}}}%%
|
||||
sequenceDiagram
|
||||
participant Test
|
||||
participant Playwright
|
||||
participant Browser
|
||||
participant API
|
||||
|
||||
rect rgb(200, 230, 201)
|
||||
Note over Test,Playwright: ✅ CORRECT: Intercept First
|
||||
Test->>Playwright: 1. waitForResponse(matcher)
|
||||
Note over Playwright: Starts listening for response
|
||||
Test->>Browser: 2. click('button')
|
||||
Browser->>API: 3. POST /api/submit
|
||||
API-->>Browser: 4. 200 OK {success: true}
|
||||
Browser-->>Playwright: 5. Response captured
|
||||
Test->>Playwright: 6. await promise
|
||||
Playwright-->>Test: 7. Returns response
|
||||
Note over Test: No race condition!
|
||||
end
|
||||
|
||||
rect rgb(255, 205, 210)
|
||||
Note over Test,API: ❌ WRONG: Action First
|
||||
Test->>Browser: 1. click('button')
|
||||
Browser->>API: 2. POST /api/submit
|
||||
API-->>Browser: 3. 200 OK (already happened!)
|
||||
Test->>Playwright: 4. waitForResponse(matcher)
|
||||
Note over Test,Playwright: Too late - response already occurred
|
||||
Note over Test: Race condition! Test hangs or fails
|
||||
end
|
||||
```
|
||||
|
||||
**Correct Order (Green):**
|
||||
1. Set up listener (`waitForResponse`)
|
||||
2. Trigger action (`click`)
|
||||
3. Wait for response (`await promise`)
|
||||
|
||||
**Wrong Order (Red):**
|
||||
1. Trigger action first
|
||||
2. Set up listener too late
|
||||
3. Response already happened - missed!
|
||||
|
||||
## How It Works in TEA
|
||||
|
||||
### TEA Generates Network-First Tests
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
// When you run *atdd or *automate, TEA generates:
|
||||
|
||||
test('should create user', async ({ page }) => {
|
||||
// TEA automatically includes network wait
|
||||
const createUserPromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/users') &&
|
||||
resp.request().method() === 'POST' &&
|
||||
resp.ok()
|
||||
);
|
||||
|
||||
await page.fill('#name', 'Test User');
|
||||
await page.click('button[type="submit"]');
|
||||
|
||||
const response = await createUserPromise;
|
||||
const user = await response.json();
|
||||
|
||||
// Validate both API and UI
|
||||
expect(user.id).toBeDefined();
|
||||
await expect(page.locator('.success')).toContainText(user.name);
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils (if `tea_use_playwright_utils: true`):**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
|
||||
test('should create user', async ({ page, interceptNetworkCall }) => {
|
||||
// TEA uses interceptNetworkCall for cleaner interception
|
||||
const createUserCall = interceptNetworkCall({
|
||||
method: 'POST',
|
||||
url: '**/api/users'
|
||||
});
|
||||
|
||||
await page.getByLabel('Name').fill('Test User');
|
||||
await page.getByRole('button', { name: 'Submit' }).click();
|
||||
|
||||
// Wait for response (automatic JSON parsing)
|
||||
const { status, responseJson: user } = await createUserCall;
|
||||
|
||||
// Validate both API and UI
|
||||
expect(status).toBe(201);
|
||||
expect(user.id).toBeDefined();
|
||||
await expect(page.locator('.success')).toContainText(user.name);
|
||||
});
|
||||
```
|
||||
|
||||
**Playwright Utils Benefits:**
|
||||
- Automatic JSON parsing (`responseJson` ready to use)
|
||||
- No manual `await response.json()`
|
||||
- Returns `{ status, responseJson }` structure
|
||||
- Cleaner, more readable code
|
||||
|
||||
### TEA Reviews for Hard Waits
|
||||
|
||||
When you run `*test-review`:
|
||||
|
||||
```markdown
|
||||
## Critical Issue: Hard Wait Detected
|
||||
|
||||
**File:** tests/e2e/submit.spec.ts:45
|
||||
**Issue:** Using `page.waitForTimeout(3000)`
|
||||
**Severity:** Critical (causes flakiness)
|
||||
|
||||
**Current Code:**
|
||||
```typescript
|
||||
await page.click('button');
|
||||
await page.waitForTimeout(3000); // ❌
|
||||
```
|
||||
|
||||
**Fix:**
|
||||
```typescript
|
||||
const responsePromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/submit') && resp.ok()
|
||||
);
|
||||
await page.click('button');
|
||||
await responsePromise; // ✅
|
||||
```
|
||||
|
||||
**Why:** Hard waits are non-deterministic. Use network-first patterns.
|
||||
```
|
||||
|
||||
## Pattern Variations
|
||||
|
||||
### Basic Response Wait
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
// Wait for any successful response
|
||||
const promise = page.waitForResponse(resp => resp.ok());
|
||||
await page.click('button');
|
||||
await promise;
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
|
||||
test('basic wait', async ({ page, interceptNetworkCall }) => {
|
||||
const responseCall = interceptNetworkCall({ url: '**' }); // Match any
|
||||
await page.click('button');
|
||||
const { status } = await responseCall;
|
||||
expect(status).toBe(200);
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Specific URL Match
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
// Wait for specific endpoint
|
||||
const promise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/users/123')
|
||||
);
|
||||
await page.goto('/user/123');
|
||||
await promise;
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
test('specific URL', async ({ page, interceptNetworkCall }) => {
|
||||
const userCall = interceptNetworkCall({ url: '**/api/users/123' });
|
||||
await page.goto('/user/123');
|
||||
const { status, responseJson } = await userCall;
|
||||
expect(status).toBe(200);
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Method + Status Match
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
// Wait for POST that returns 201
|
||||
const promise = page.waitForResponse(
|
||||
resp =>
|
||||
resp.url().includes('/api/users') &&
|
||||
resp.request().method() === 'POST' &&
|
||||
resp.status() === 201
|
||||
);
|
||||
await page.click('button[type="submit"]');
|
||||
await promise;
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
test('method and status', async ({ page, interceptNetworkCall }) => {
|
||||
const createCall = interceptNetworkCall({
|
||||
method: 'POST',
|
||||
url: '**/api/users'
|
||||
});
|
||||
await page.click('button[type="submit"]');
|
||||
const { status, responseJson } = await createCall;
|
||||
expect(status).toBe(201); // Explicit status check
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Multiple Responses
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
// Wait for multiple API calls
|
||||
const [usersResp, postsResp] = await Promise.all([
|
||||
page.waitForResponse(resp => resp.url().includes('/api/users')),
|
||||
page.waitForResponse(resp => resp.url().includes('/api/posts')),
|
||||
page.goto('/dashboard') // Triggers both requests
|
||||
]);
|
||||
|
||||
const users = await usersResp.json();
|
||||
const posts = await postsResp.json();
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
test('multiple responses', async ({ page, interceptNetworkCall }) => {
|
||||
const usersCall = interceptNetworkCall({ url: '**/api/users' });
|
||||
const postsCall = interceptNetworkCall({ url: '**/api/posts' });
|
||||
|
||||
await page.goto('/dashboard'); // Triggers both
|
||||
|
||||
const [{ responseJson: users }, { responseJson: posts }] = await Promise.all([
|
||||
usersCall,
|
||||
postsCall
|
||||
]);
|
||||
|
||||
expect(users).toBeInstanceOf(Array);
|
||||
expect(posts).toBeInstanceOf(Array);
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Validate Response Data
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
// Verify API response before asserting UI
|
||||
const promise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/checkout') && resp.ok()
|
||||
);
|
||||
|
||||
await page.click('button:has-text("Complete Order")');
|
||||
|
||||
const response = await promise;
|
||||
const order = await response.json();
|
||||
|
||||
// Response validation
|
||||
expect(order.status).toBe('confirmed');
|
||||
expect(order.total).toBeGreaterThan(0);
|
||||
|
||||
// UI validation
|
||||
await expect(page.locator('.order-confirmation')).toContainText(order.id);
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
test('validate response data', async ({ page, interceptNetworkCall }) => {
|
||||
const checkoutCall = interceptNetworkCall({
|
||||
method: 'POST',
|
||||
url: '**/api/checkout'
|
||||
});
|
||||
|
||||
await page.click('button:has-text("Complete Order")');
|
||||
|
||||
const { status, responseJson: order } = await checkoutCall;
|
||||
|
||||
// Response validation (automatic JSON parsing)
|
||||
expect(status).toBe(200);
|
||||
expect(order.status).toBe('confirmed');
|
||||
expect(order.total).toBeGreaterThan(0);
|
||||
|
||||
// UI validation
|
||||
await expect(page.locator('.order-confirmation')).toContainText(order.id);
|
||||
});
|
||||
```
|
||||
|
||||
## Advanced Patterns
|
||||
|
||||
### HAR Recording for Offline Testing
|
||||
|
||||
**Vanilla Playwright (Manual HAR Handling):**
|
||||
|
||||
```typescript
|
||||
// First run: Record mode (saves HAR file)
|
||||
test('offline testing - RECORD', async ({ page, context }) => {
|
||||
// Record mode: Save network traffic to HAR
|
||||
await context.routeFromHAR('./hars/dashboard.har', {
|
||||
url: '**/api/**',
|
||||
update: true // Update HAR file
|
||||
});
|
||||
|
||||
await page.goto('/dashboard');
|
||||
// All network traffic saved to dashboard.har
|
||||
});
|
||||
|
||||
// Subsequent runs: Playback mode (uses saved HAR)
|
||||
test('offline testing - PLAYBACK', async ({ page, context }) => {
|
||||
// Playback mode: Use saved network traffic
|
||||
await context.routeFromHAR('./hars/dashboard.har', {
|
||||
url: '**/api/**',
|
||||
update: false // Use existing HAR, no network calls
|
||||
});
|
||||
|
||||
await page.goto('/dashboard');
|
||||
// Uses recorded responses, no backend needed
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils (Automatic HAR Management):**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/network-recorder/fixtures';
|
||||
|
||||
// Record mode: Set environment variable
|
||||
process.env.PW_NET_MODE = 'record';
|
||||
|
||||
test('should work offline', async ({ page, context, networkRecorder }) => {
|
||||
await networkRecorder.setup(context); // Handles HAR automatically
|
||||
|
||||
await page.goto('/dashboard');
|
||||
await page.click('#add-item');
|
||||
// All network traffic recorded, CRUD operations detected
|
||||
});
|
||||
```
|
||||
|
||||
**Switch to playback:**
|
||||
```bash
|
||||
# Playback mode (offline)
|
||||
PW_NET_MODE=playback npx playwright test
|
||||
# Uses HAR file, no backend needed!
|
||||
```
|
||||
|
||||
**Playwright Utils Benefits:**
|
||||
- Automatic HAR file management (naming, paths)
|
||||
- CRUD operation detection (stateful mocking)
|
||||
- Environment variable control (easy switching)
|
||||
- Works for complex interactions (create, update, delete)
|
||||
- No manual route configuration
|
||||
|
||||
### Network Request Interception
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
test('should handle API error', async ({ page }) => {
|
||||
// Manual route setup
|
||||
await page.route('**/api/users', (route) => {
|
||||
route.fulfill({
|
||||
status: 500,
|
||||
body: JSON.stringify({ error: 'Internal server error' })
|
||||
});
|
||||
});
|
||||
|
||||
await page.goto('/users');
|
||||
|
||||
const response = await page.waitForResponse('**/api/users');
|
||||
const error = await response.json();
|
||||
|
||||
expect(error.error).toContain('Internal server');
|
||||
await expect(page.locator('.error-message')).toContainText('Server error');
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
|
||||
test('should handle API error', async ({ page, interceptNetworkCall }) => {
|
||||
// Stub API to return error (set up BEFORE navigation)
|
||||
const usersCall = interceptNetworkCall({
|
||||
method: 'GET',
|
||||
url: '**/api/users',
|
||||
fulfillResponse: {
|
||||
status: 500,
|
||||
body: { error: 'Internal server error' }
|
||||
}
|
||||
});
|
||||
|
||||
await page.goto('/users');
|
||||
|
||||
// Wait for mocked response and access parsed data
|
||||
const { status, responseJson } = await usersCall;
|
||||
|
||||
expect(status).toBe(500);
|
||||
expect(responseJson.error).toContain('Internal server');
|
||||
await expect(page.locator('.error-message')).toContainText('Server error');
|
||||
});
|
||||
```
|
||||
|
||||
**Playwright Utils Benefits:**
|
||||
- Automatic JSON parsing (`responseJson` ready to use)
|
||||
- Returns promise with `{ status, responseJson, requestJson }`
|
||||
- No need to pass `page` (auto-injected by fixture)
|
||||
- Glob pattern matching (simpler than regex)
|
||||
- Single declarative call (setup + wait in one)
|
||||
|
||||
## Comparison: Traditional vs Network-First
|
||||
|
||||
### Loading Dashboard Data
|
||||
|
||||
**Traditional (Flaky):**
|
||||
```typescript
|
||||
test('dashboard loads data', async ({ page }) => {
|
||||
await page.goto('/dashboard');
|
||||
await page.waitForTimeout(2000); // ❌ Magic number
|
||||
await expect(page.locator('table tr')).toHaveCount(5);
|
||||
});
|
||||
```
|
||||
|
||||
**Failure modes:**
|
||||
- API takes 2.5s → test fails
|
||||
- API returns 3 items not 5 → hard to debug (which issue?)
|
||||
- CI slower than local → fails in CI only
|
||||
|
||||
**Network-First (Deterministic):**
|
||||
```typescript
|
||||
test('dashboard loads data', async ({ page }) => {
|
||||
const apiPromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/dashboard') && resp.ok()
|
||||
);
|
||||
|
||||
await page.goto('/dashboard');
|
||||
|
||||
const response = await apiPromise;
|
||||
const { items } = await response.json();
|
||||
|
||||
// Validate API response
|
||||
expect(items).toHaveLength(5);
|
||||
|
||||
// Validate UI matches API
|
||||
await expect(page.locator('table tr')).toHaveCount(items.length);
|
||||
});
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Waits exactly as long as needed (100ms or 5s, doesn't matter)
|
||||
- Validates API response (catch backend errors)
|
||||
- Validates UI matches API (catch frontend bugs)
|
||||
- Works in any environment (local, CI, staging)
|
||||
|
||||
**With Playwright Utils (Even Better):**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
|
||||
test('dashboard loads data', async ({ page, interceptNetworkCall }) => {
|
||||
const dashboardCall = interceptNetworkCall({
|
||||
method: 'GET',
|
||||
url: '**/api/dashboard'
|
||||
});
|
||||
|
||||
await page.goto('/dashboard');
|
||||
|
||||
const { status, responseJson: { items } } = await dashboardCall;
|
||||
|
||||
// Validate API response (automatic JSON parsing)
|
||||
expect(status).toBe(200);
|
||||
expect(items).toHaveLength(5);
|
||||
|
||||
// Validate UI matches API
|
||||
await expect(page.locator('table tr')).toHaveCount(items.length);
|
||||
});
|
||||
```
|
||||
|
||||
**Additional Benefits:**
|
||||
- No manual `await response.json()` (automatic parsing)
|
||||
- Cleaner destructuring of nested data
|
||||
- Consistent API across all network calls
|
||||
|
||||
---
|
||||
|
||||
### Form Submission
|
||||
|
||||
**Traditional (Flaky):**
|
||||
```typescript
|
||||
test('form submission', async ({ page }) => {
|
||||
await page.fill('#email', 'test@example.com');
|
||||
await page.click('button[type="submit"]');
|
||||
await page.waitForTimeout(3000); // ❌ Hope it's enough
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Network-First (Deterministic):**
|
||||
```typescript
|
||||
test('form submission', async ({ page }) => {
|
||||
const submitPromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/submit') &&
|
||||
resp.request().method() === 'POST' &&
|
||||
resp.ok()
|
||||
);
|
||||
|
||||
await page.fill('#email', 'test@example.com');
|
||||
await page.click('button[type="submit"]');
|
||||
|
||||
const response = await submitPromise;
|
||||
const result = await response.json();
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
|
||||
test('form submission', async ({ page, interceptNetworkCall }) => {
|
||||
const submitCall = interceptNetworkCall({
|
||||
method: 'POST',
|
||||
url: '**/api/submit'
|
||||
});
|
||||
|
||||
await page.getByLabel('Email').fill('test@example.com');
|
||||
await page.getByRole('button', { name: 'Submit' }).click();
|
||||
|
||||
const { status, responseJson: result } = await submitCall;
|
||||
|
||||
// Automatic JSON parsing, no manual await
|
||||
expect(status).toBe(200);
|
||||
expect(result.success).toBe(true);
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Progression:**
|
||||
- Traditional: Hard waits (flaky)
|
||||
- Network-First (Vanilla): waitForResponse (deterministic)
|
||||
- Network-First (PW-Utils): interceptNetworkCall (deterministic + cleaner API)
|
||||
|
||||
---
|
||||
|
||||
## Common Misconceptions
|
||||
|
||||
### "I Already Use waitForSelector"
|
||||
|
||||
```typescript
|
||||
// This is still a hard wait in disguise
|
||||
await page.click('button');
|
||||
await page.waitForSelector('.success', { timeout: 5000 });
|
||||
```
|
||||
|
||||
**Problem:** Waiting for DOM, not for the API that caused DOM change.
|
||||
|
||||
**Better:**
|
||||
```typescript
|
||||
await page.waitForResponse(matcher); // Wait for root cause
|
||||
await page.waitForSelector('.success'); // Then validate UI
|
||||
```
|
||||
|
||||
### "My Tests Are Fast, Why Add Complexity?"
|
||||
|
||||
**Short-term:** Tests are fast locally
|
||||
|
||||
**Long-term problems:**
|
||||
- Different environments (CI slower)
|
||||
- Under load (API slower)
|
||||
- Network variability (random)
|
||||
- Scaling test suite (100 → 1000 tests)
|
||||
|
||||
**Network-first prevents these issues before they appear.**
|
||||
|
||||
### "Too Much Boilerplate"
|
||||
|
||||
**Problem:** `waitForResponse` is verbose, repeated in every test.
|
||||
|
||||
**Solution:** Use Playwright Utils `interceptNetworkCall` - built-in fixture that reduces boilerplate.
|
||||
|
||||
**Vanilla Playwright (Repetitive):**
|
||||
```typescript
|
||||
test('test 1', async ({ page }) => {
|
||||
const promise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/submit') && resp.ok()
|
||||
);
|
||||
await page.click('button');
|
||||
await promise;
|
||||
});
|
||||
|
||||
test('test 2', async ({ page }) => {
|
||||
const promise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/load') && resp.ok()
|
||||
);
|
||||
await page.click('button');
|
||||
await promise;
|
||||
});
|
||||
// Repeated pattern in every test
|
||||
```
|
||||
|
||||
**With Playwright Utils (Cleaner):**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
|
||||
test('test 1', async ({ page, interceptNetworkCall }) => {
|
||||
const submitCall = interceptNetworkCall({ url: '**/api/submit' });
|
||||
await page.click('button');
|
||||
const { status, responseJson } = await submitCall;
|
||||
expect(status).toBe(200);
|
||||
});
|
||||
|
||||
test('test 2', async ({ page, interceptNetworkCall }) => {
|
||||
const loadCall = interceptNetworkCall({ url: '**/api/load' });
|
||||
await page.click('button');
|
||||
const { responseJson } = await loadCall;
|
||||
// Automatic JSON parsing, cleaner API
|
||||
});
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Less boilerplate (fixture handles complexity)
|
||||
- Automatic JSON parsing
|
||||
- Glob pattern matching (`**/api/**`)
|
||||
- Consistent API across all tests
|
||||
|
||||
See [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md#intercept-network-call) for setup.
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
For detailed network-first patterns, see the knowledge base:
|
||||
- [Knowledge Base Index - Network & Reliability](/docs/reference/tea/knowledge-base.md)
|
||||
- [Complete Knowledge Base Index](/docs/reference/tea/knowledge-base.md)
|
||||
|
||||
## Related Concepts
|
||||
|
||||
**Core TEA Concepts:**
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Determinism requires network-first
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - High-risk features need reliable tests
|
||||
|
||||
**Technical Patterns:**
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Network utilities as fixtures
|
||||
- [Knowledge Base System](/docs/explanation/tea/knowledge-base-system.md) - Network patterns in knowledge base
|
||||
|
||||
**Overview:**
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Network-first in workflows
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - Why flakiness matters
|
||||
|
||||
## Practical Guides
|
||||
|
||||
**Workflow Guides:**
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md) - Review for hard waits
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Generate network-first tests
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Expand with network patterns
|
||||
|
||||
**Use-Case Guides:**
|
||||
- [Using TEA with Existing Tests](/docs/how-to/brownfield/use-tea-with-existing-tests.md) - Fix flaky legacy tests
|
||||
|
||||
**Customization:**
|
||||
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) - Network utilities (recorder, interceptor, error monitor)
|
||||
|
||||
## Reference
|
||||
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - All workflows use network-first
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Network-first fragment
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - Network-first pattern term
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,586 @@
|
|||
---
|
||||
title: "Risk-Based Testing Explained"
|
||||
description: Understanding how TEA uses probability × impact scoring to prioritize testing effort
|
||||
---
|
||||
|
||||
# Risk-Based Testing Explained
|
||||
|
||||
Risk-based testing is TEA's core principle: testing depth scales with business impact. Instead of testing everything equally, focus effort where failures hurt most.
|
||||
|
||||
## Overview
|
||||
|
||||
Traditional testing approaches treat all features equally:
|
||||
- Every feature gets same test coverage
|
||||
- Same level of scrutiny regardless of impact
|
||||
- No systematic prioritization
|
||||
- Testing becomes checkbox exercise
|
||||
|
||||
**Risk-based testing asks:**
|
||||
- What's the probability this will fail?
|
||||
- What's the impact if it does fail?
|
||||
- How much testing is appropriate for this risk level?
|
||||
|
||||
**Result:** Testing effort matches business criticality.
|
||||
|
||||
## The Problem
|
||||
|
||||
### Equal Testing for Unequal Risk
|
||||
|
||||
```markdown
|
||||
Feature A: User login (critical path, millions of users)
|
||||
Feature B: Export to PDF (nice-to-have, rarely used)
|
||||
|
||||
Traditional approach:
|
||||
- Both get 10 tests
|
||||
- Both get same review scrutiny
|
||||
- Both take same development time
|
||||
|
||||
Problem: Wasting effort on low-impact features while under-testing critical paths.
|
||||
```
|
||||
|
||||
### No Objective Prioritization
|
||||
|
||||
```markdown
|
||||
PM: "We need more tests for checkout"
|
||||
QA: "How many tests?"
|
||||
PM: "I don't know... a lot?"
|
||||
QA: "How do we know when we have enough?"
|
||||
PM: "When it feels safe?"
|
||||
|
||||
Problem: Subjective decisions, no data, political debates.
|
||||
```
|
||||
|
||||
## The Solution: Probability × Impact Scoring
|
||||
|
||||
### Risk Score = Probability × Impact
|
||||
|
||||
**Probability** (How likely to fail?)
|
||||
- **1 (Low):** Stable, well-tested, simple logic
|
||||
- **2 (Medium):** Moderate complexity, some unknowns
|
||||
- **3 (High):** Complex, untested, many edge cases
|
||||
|
||||
**Impact** (How bad if it fails?)
|
||||
- **1 (Low):** Minor inconvenience, few users affected
|
||||
- **2 (Medium):** Degraded experience, workarounds exist
|
||||
- **3 (High):** Critical path broken, business impact
|
||||
|
||||
**Score Range:** 1-9
|
||||
|
||||
#### Risk Scoring Matrix
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'14px'}}}%%
|
||||
graph TD
|
||||
subgraph Matrix[" "]
|
||||
direction TB
|
||||
subgraph Impact3["Impact: HIGH (3)"]
|
||||
P1I3["Score: 3<br/>Low Risk"]
|
||||
P2I3["Score: 6<br/>HIGH RISK<br/>Mitigation Required"]
|
||||
P3I3["Score: 9<br/>CRITICAL<br/>Blocks Release"]
|
||||
end
|
||||
subgraph Impact2["Impact: MEDIUM (2)"]
|
||||
P1I2["Score: 2<br/>Low Risk"]
|
||||
P2I2["Score: 4<br/>Medium Risk"]
|
||||
P3I2["Score: 6<br/>HIGH RISK<br/>Mitigation Required"]
|
||||
end
|
||||
subgraph Impact1["Impact: LOW (1)"]
|
||||
P1I1["Score: 1<br/>Low Risk"]
|
||||
P2I1["Score: 2<br/>Low Risk"]
|
||||
P3I1["Score: 3<br/>Low Risk"]
|
||||
end
|
||||
end
|
||||
|
||||
Prob1["Probability: LOW (1)"] -.-> P1I1
|
||||
Prob1 -.-> P1I2
|
||||
Prob1 -.-> P1I3
|
||||
|
||||
Prob2["Probability: MEDIUM (2)"] -.-> P2I1
|
||||
Prob2 -.-> P2I2
|
||||
Prob2 -.-> P2I3
|
||||
|
||||
Prob3["Probability: HIGH (3)"] -.-> P3I1
|
||||
Prob3 -.-> P3I2
|
||||
Prob3 -.-> P3I3
|
||||
|
||||
style P3I3 fill:#f44336,stroke:#b71c1c,stroke-width:3px,color:#fff
|
||||
style P2I3 fill:#ff9800,stroke:#e65100,stroke-width:2px,color:#000
|
||||
style P3I2 fill:#ff9800,stroke:#e65100,stroke-width:2px,color:#000
|
||||
style P2I2 fill:#fff9c4,stroke:#f57f17,stroke-width:1px,color:#000
|
||||
style P1I1 fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px,color:#000
|
||||
style P2I1 fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px,color:#000
|
||||
style P3I1 fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px,color:#000
|
||||
style P1I2 fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px,color:#000
|
||||
style P1I3 fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px,color:#000
|
||||
```
|
||||
|
||||
**Legend:**
|
||||
- 🔴 Red (Score 9): CRITICAL - Blocks release
|
||||
- 🟠 Orange (Score 6-8): HIGH RISK - Mitigation required
|
||||
- 🟡 Yellow (Score 4-5): MEDIUM - Mitigation recommended
|
||||
- 🟢 Green (Score 1-3): LOW - Optional mitigation
|
||||
|
||||
### Scoring Examples
|
||||
|
||||
**Score 9 (Critical):**
|
||||
```
|
||||
Feature: Payment processing
|
||||
Probability: 3 (complex third-party integration)
|
||||
Impact: 3 (broken payments = lost revenue)
|
||||
Score: 3 × 3 = 9
|
||||
|
||||
Action: Extensive testing required
|
||||
- E2E tests for all payment flows
|
||||
- API tests for all payment scenarios
|
||||
- Error handling for all failure modes
|
||||
- Security testing for payment data
|
||||
- Load testing for high traffic
|
||||
- Monitoring and alerts
|
||||
```
|
||||
|
||||
**Score 1 (Low):**
|
||||
```
|
||||
Feature: Change profile theme color
|
||||
Probability: 1 (simple UI toggle)
|
||||
Impact: 1 (cosmetic only)
|
||||
Score: 1 × 1 = 1
|
||||
|
||||
Action: Minimal testing
|
||||
- One E2E smoke test
|
||||
- Skip edge cases
|
||||
- No API tests needed
|
||||
```
|
||||
|
||||
**Score 6 (Medium-High):**
|
||||
```
|
||||
Feature: User profile editing
|
||||
Probability: 2 (moderate complexity)
|
||||
Impact: 3 (users can't update info)
|
||||
Score: 2 × 3 = 6
|
||||
|
||||
Action: Focused testing
|
||||
- E2E test for happy path
|
||||
- API tests for CRUD operations
|
||||
- Validation testing
|
||||
- Skip low-value edge cases
|
||||
```
|
||||
|
||||
## How It Works in TEA
|
||||
|
||||
### 1. Risk Categories
|
||||
|
||||
TEA assesses risk across 6 categories:
|
||||
|
||||
**TECH** - Technical debt, architecture fragility
|
||||
```
|
||||
Example: Migrating from REST to GraphQL
|
||||
Probability: 3 (major architectural change)
|
||||
Impact: 3 (affects all API consumers)
|
||||
Score: 9 - Extensive integration testing required
|
||||
```
|
||||
|
||||
**SEC** - Security vulnerabilities
|
||||
```
|
||||
Example: Adding OAuth integration
|
||||
Probability: 2 (third-party dependency)
|
||||
Impact: 3 (auth breach = data exposure)
|
||||
Score: 6 - Security testing mandatory
|
||||
```
|
||||
|
||||
**PERF** - Performance degradation
|
||||
```
|
||||
Example: Adding real-time notifications
|
||||
Probability: 2 (WebSocket complexity)
|
||||
Impact: 2 (slower experience)
|
||||
Score: 4 - Load testing recommended
|
||||
```
|
||||
|
||||
**DATA** - Data integrity, corruption
|
||||
```
|
||||
Example: Database migration
|
||||
Probability: 2 (schema changes)
|
||||
Impact: 3 (data loss unacceptable)
|
||||
Score: 6 - Data validation tests required
|
||||
```
|
||||
|
||||
**BUS** - Business logic errors
|
||||
```
|
||||
Example: Discount calculation
|
||||
Probability: 2 (business rules complex)
|
||||
Impact: 3 (wrong prices = revenue loss)
|
||||
Score: 6 - Business logic tests mandatory
|
||||
```
|
||||
|
||||
**OPS** - Operational issues
|
||||
```
|
||||
Example: Logging system update
|
||||
Probability: 1 (straightforward)
|
||||
Impact: 2 (debugging harder without logs)
|
||||
Score: 2 - Basic smoke test sufficient
|
||||
```
|
||||
|
||||
### 2. Test Priorities (P0-P3)
|
||||
|
||||
Risk scores inform test priorities (but aren't the only factor):
|
||||
|
||||
**P0 - Critical Path**
|
||||
- **Risk Scores:** Typically 6-9 (high risk)
|
||||
- **Other Factors:** Revenue impact, security-critical, regulatory compliance, frequent usage
|
||||
- **Coverage Target:** 100%
|
||||
- **Test Levels:** E2E + API
|
||||
- **Example:** Login, checkout, payment processing
|
||||
|
||||
**P1 - High Value**
|
||||
- **Risk Scores:** Typically 4-6 (medium-high risk)
|
||||
- **Other Factors:** Core user journeys, complex logic, integration points
|
||||
- **Coverage Target:** 90%
|
||||
- **Test Levels:** API + selective E2E
|
||||
- **Example:** Profile editing, search, filters
|
||||
|
||||
**P2 - Medium Value**
|
||||
- **Risk Scores:** Typically 2-4 (medium risk)
|
||||
- **Other Factors:** Secondary features, admin functionality, reporting
|
||||
- **Coverage Target:** 50%
|
||||
- **Test Levels:** API happy path only
|
||||
- **Example:** Export features, advanced settings
|
||||
|
||||
**P3 - Low Value**
|
||||
- **Risk Scores:** Typically 1-2 (low risk)
|
||||
- **Other Factors:** Rarely used, nice-to-have, cosmetic
|
||||
- **Coverage Target:** 20% (smoke test)
|
||||
- **Test Levels:** E2E smoke test only
|
||||
- **Example:** Theme customization, experimental features
|
||||
|
||||
**Note:** Priorities consider risk scores plus business context (usage frequency, user impact, etc.). See [Test Priorities Matrix](/docs/reference/tea/knowledge-base.md#quality-standards) for complete criteria.
|
||||
|
||||
### 3. Mitigation Plans
|
||||
|
||||
**Scores ≥6 require documented mitigation:**
|
||||
|
||||
```markdown
|
||||
## Risk Mitigation
|
||||
|
||||
**Risk:** Payment integration failure (Score: 9)
|
||||
|
||||
**Mitigation Plan:**
|
||||
- Create comprehensive test suite (20+ tests)
|
||||
- Add payment sandbox environment
|
||||
- Implement retry logic with idempotency
|
||||
- Add monitoring and alerts
|
||||
- Document rollback procedure
|
||||
|
||||
**Owner:** Backend team lead
|
||||
**Deadline:** Before production deployment
|
||||
**Status:** In progress
|
||||
```
|
||||
|
||||
**Gate Rules:**
|
||||
- **Score = 9** (Critical): Mandatory FAIL - blocks release without mitigation
|
||||
- **Score 6-8** (High): Requires mitigation plan, becomes CONCERNS if incomplete
|
||||
- **Score 4-5** (Medium): Mitigation recommended but not required
|
||||
- **Score 1-3** (Low): No mitigation needed
|
||||
|
||||
## Comparison: Traditional vs Risk-Based
|
||||
|
||||
### Traditional Approach
|
||||
|
||||
```typescript
|
||||
// Test everything equally
|
||||
describe('User profile', () => {
|
||||
test('should display name');
|
||||
test('should display email');
|
||||
test('should display phone');
|
||||
test('should display address');
|
||||
test('should display bio');
|
||||
test('should display avatar');
|
||||
test('should display join date');
|
||||
test('should display last login');
|
||||
test('should display theme preference');
|
||||
test('should display language preference');
|
||||
// 10 tests for profile display (all equal priority)
|
||||
});
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Same effort for critical (name) vs trivial (theme)
|
||||
- No guidance on what matters
|
||||
- Wastes time on low-value tests
|
||||
|
||||
### Risk-Based Approach
|
||||
|
||||
```typescript
|
||||
// Test based on risk
|
||||
|
||||
describe('User profile - Critical (P0)', () => {
|
||||
test('should display name and email'); // Score: 9 (identity critical)
|
||||
test('should allow editing name and email');
|
||||
test('should validate email format');
|
||||
test('should prevent unauthorized edits');
|
||||
// 4 focused tests on high-risk areas
|
||||
});
|
||||
|
||||
describe('User profile - High Value (P1)', () => {
|
||||
test('should upload avatar'); // Score: 6 (users care about this)
|
||||
test('should update bio');
|
||||
// 2 tests for high-value features
|
||||
});
|
||||
|
||||
// P2: Theme preference - single smoke test
|
||||
// P3: Last login display - skip (read-only, low value)
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- 6 focused tests vs 10 unfocused tests
|
||||
- Effort matches business impact
|
||||
- Clear priorities guide development
|
||||
- No wasted effort on trivial features
|
||||
|
||||
## When to Use Risk-Based Testing
|
||||
|
||||
### Always Use For:
|
||||
|
||||
**Enterprise projects:**
|
||||
- High stakes (revenue, compliance, security)
|
||||
- Many features competing for test effort
|
||||
- Need objective prioritization
|
||||
|
||||
**Large codebases:**
|
||||
- Can't test everything exhaustively
|
||||
- Need to focus limited QA resources
|
||||
- Want data-driven decisions
|
||||
|
||||
**Regulated industries:**
|
||||
- Must justify testing decisions
|
||||
- Auditors want risk assessments
|
||||
- Compliance requires evidence
|
||||
|
||||
### Consider Skipping For:
|
||||
|
||||
**Tiny projects:**
|
||||
- 5 features total
|
||||
- Can test everything thoroughly
|
||||
- Risk scoring is overhead
|
||||
|
||||
**Prototypes:**
|
||||
- Throw-away code
|
||||
- Speed over quality
|
||||
- Learning experiments
|
||||
|
||||
## Real-World Example
|
||||
|
||||
### Scenario: E-Commerce Checkout Redesign
|
||||
|
||||
**Feature:** Redesigning checkout flow from 5 steps to 3 steps
|
||||
|
||||
**Risk Assessment:**
|
||||
|
||||
| Component | Probability | Impact | Score | Priority | Testing |
|
||||
|-----------|-------------|--------|-------|----------|---------|
|
||||
| **Payment processing** | 3 | 3 | 9 | P0 | 15 E2E + 20 API tests |
|
||||
| **Order validation** | 2 | 3 | 6 | P1 | 5 E2E + 10 API tests |
|
||||
| **Shipping calculation** | 2 | 2 | 4 | P1 | 3 E2E + 8 API tests |
|
||||
| **Promo code validation** | 2 | 2 | 4 | P1 | 2 E2E + 5 API tests |
|
||||
| **Gift message** | 1 | 1 | 1 | P3 | 1 E2E smoke test |
|
||||
|
||||
**Test Budget:** 40 hours
|
||||
|
||||
**Allocation:**
|
||||
- Payment (Score 9): 20 hours (50%)
|
||||
- Order validation (Score 6): 8 hours (20%)
|
||||
- Shipping (Score 4): 6 hours (15%)
|
||||
- Promo codes (Score 4): 4 hours (10%)
|
||||
- Gift message (Score 1): 2 hours (5%)
|
||||
|
||||
**Result:** 50% of effort on highest-risk feature (payment), proportional allocation for others.
|
||||
|
||||
### Without Risk-Based Testing:
|
||||
|
||||
**Equal allocation:** 8 hours per component = wasted effort on gift message, under-testing payment.
|
||||
|
||||
**Result:** Payment bugs slip through (critical), perfect testing of gift message (trivial).
|
||||
|
||||
## Mitigation Strategies by Risk Level
|
||||
|
||||
### Score 9: Mandatory Mitigation (Blocks Release)
|
||||
|
||||
```markdown
|
||||
**Gate Impact:** FAIL - Cannot deploy without mitigation
|
||||
|
||||
**Actions:**
|
||||
- Comprehensive test suite (E2E, API, security)
|
||||
- Multiple test environments (dev, staging, prod-mirror)
|
||||
- Load testing and performance validation
|
||||
- Security audit and penetration testing
|
||||
- Monitoring and alerting
|
||||
- Rollback plan documented
|
||||
- On-call rotation assigned
|
||||
|
||||
**Cannot deploy until score is mitigated below 9.**
|
||||
```
|
||||
|
||||
### Score 6-8: Required Mitigation (Gate: CONCERNS)
|
||||
|
||||
```markdown
|
||||
**Gate Impact:** CONCERNS - Can deploy with documented mitigation plan
|
||||
|
||||
**Actions:**
|
||||
- Targeted test suite (happy path + critical errors)
|
||||
- Test environment setup
|
||||
- Monitoring plan
|
||||
- Document mitigation and owners
|
||||
|
||||
**Can deploy with approved mitigation plan.**
|
||||
```
|
||||
|
||||
### Score 4-5: Recommended Mitigation
|
||||
|
||||
```markdown
|
||||
**Gate Impact:** Advisory - Does not affect gate decision
|
||||
|
||||
**Actions:**
|
||||
- Basic test coverage
|
||||
- Standard monitoring
|
||||
- Document known limitations
|
||||
|
||||
**Can deploy, mitigation recommended but not required.**
|
||||
```
|
||||
|
||||
### Score 1-3: Optional Mitigation
|
||||
|
||||
```markdown
|
||||
**Gate Impact:** None
|
||||
|
||||
**Actions:**
|
||||
- Smoke test if desired
|
||||
- Feature flag for easy disable (optional)
|
||||
|
||||
**Can deploy without mitigation.**
|
||||
```
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
For detailed risk governance patterns, see the knowledge base:
|
||||
- [Knowledge Base Index - Risk & Gates](/docs/reference/tea/knowledge-base.md)
|
||||
- [TEA Command Reference - *test-design](/docs/reference/tea/commands.md#test-design)
|
||||
|
||||
### Risk Scoring Matrix
|
||||
|
||||
TEA uses this framework in `*test-design`:
|
||||
|
||||
```
|
||||
Impact
|
||||
1 2 3
|
||||
┌────┬────┬────┐
|
||||
1 │ 1 │ 2 │ 3 │ Low risk
|
||||
P 2 │ 2 │ 4 │ 6 │ Medium risk
|
||||
r 3 │ 3 │ 6 │ 9 │ High risk
|
||||
o └────┴────┴────┘
|
||||
b Low Med High
|
||||
```
|
||||
|
||||
### Gate Decision Rules
|
||||
|
||||
| Score | Mitigation Required | Gate Impact |
|
||||
|-------|-------------------|-------------|
|
||||
| **9** | Mandatory, blocks release | FAIL if no mitigation |
|
||||
| **6-8** | Required, documented plan | CONCERNS if incomplete |
|
||||
| **4-5** | Recommended | Advisory only |
|
||||
| **1-3** | Optional | No impact |
|
||||
|
||||
#### Gate Decision Flow
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'14px'}}}%%
|
||||
flowchart TD
|
||||
Start([Risk Assessment]) --> Score{Risk Score?}
|
||||
|
||||
Score -->|Score = 9| Critical[CRITICAL RISK<br/>Score: 9]
|
||||
Score -->|Score 6-8| High[HIGH RISK<br/>Score: 6-8]
|
||||
Score -->|Score 4-5| Medium[MEDIUM RISK<br/>Score: 4-5]
|
||||
Score -->|Score 1-3| Low[LOW RISK<br/>Score: 1-3]
|
||||
|
||||
Critical --> HasMit9{Mitigation<br/>Plan?}
|
||||
HasMit9 -->|Yes| Concerns9[CONCERNS ⚠️<br/>Can deploy with plan]
|
||||
HasMit9 -->|No| Fail[FAIL ❌<br/>Blocks release]
|
||||
|
||||
High --> HasMit6{Mitigation<br/>Plan?}
|
||||
HasMit6 -->|Yes| Pass6[PASS ✅<br/>or CONCERNS ⚠️]
|
||||
HasMit6 -->|No| Concerns6[CONCERNS ⚠️<br/>Document plan needed]
|
||||
|
||||
Medium --> Advisory[Advisory Only<br/>No gate impact]
|
||||
Low --> NoAction[No Action<br/>Proceed]
|
||||
|
||||
style Critical fill:#f44336,stroke:#b71c1c,stroke-width:3px,color:#fff
|
||||
style Fail fill:#d32f2f,stroke:#b71c1c,stroke-width:3px,color:#fff
|
||||
style High fill:#ff9800,stroke:#e65100,stroke-width:2px,color:#000
|
||||
style Concerns9 fill:#ffc107,stroke:#f57f17,stroke-width:2px,color:#000
|
||||
style Concerns6 fill:#ffc107,stroke:#f57f17,stroke-width:2px,color:#000
|
||||
style Pass6 fill:#4caf50,stroke:#1b5e20,stroke-width:2px,color:#fff
|
||||
style Medium fill:#fff9c4,stroke:#f57f17,stroke-width:1px,color:#000
|
||||
style Low fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px,color:#000
|
||||
style Advisory fill:#e8f5e9,stroke:#2e7d32,stroke-width:1px,color:#000
|
||||
style NoAction fill:#e8f5e9,stroke:#2e7d32,stroke-width:1px,color:#000
|
||||
```
|
||||
|
||||
## Common Misconceptions
|
||||
|
||||
### "Risk-based = Less Testing"
|
||||
|
||||
**Wrong:** Risk-based testing often means MORE testing where it matters.
|
||||
|
||||
**Example:**
|
||||
- Traditional: 50 tests spread equally
|
||||
- Risk-based: 70 tests focused on P0/P1 (more total, better allocated)
|
||||
|
||||
### "Low Priority = Skip Testing"
|
||||
|
||||
**Wrong:** P3 still gets smoke tests.
|
||||
|
||||
**Correct:**
|
||||
- P3: Smoke test (feature works at all)
|
||||
- P2: Happy path (feature works correctly)
|
||||
- P1: Happy path + errors
|
||||
- P0: Comprehensive (all scenarios)
|
||||
|
||||
### "Risk Scores Are Permanent"
|
||||
|
||||
**Wrong:** Risk changes over time.
|
||||
|
||||
**Correct:**
|
||||
- Initial launch: Payment is Score 9 (untested integration)
|
||||
- After 6 months: Payment is Score 6 (proven in production)
|
||||
- Re-assess risk quarterly
|
||||
|
||||
## Related Concepts
|
||||
|
||||
**Core TEA Concepts:**
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Quality complements risk assessment
|
||||
- [Engagement Models](/docs/explanation/tea/engagement-models.md) - When risk-based testing matters most
|
||||
- [Knowledge Base System](/docs/explanation/tea/knowledge-base-system.md) - How risk patterns are loaded
|
||||
|
||||
**Technical Patterns:**
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Building risk-appropriate test infrastructure
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Quality patterns for high-risk features
|
||||
|
||||
**Overview:**
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Risk assessment in TEA lifecycle
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - Design philosophy
|
||||
|
||||
## Practical Guides
|
||||
|
||||
**Workflow Guides:**
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md) - Apply risk scoring
|
||||
- [How to Run Trace](/docs/how-to/workflows/run-trace.md) - Gate decisions based on risk
|
||||
- [How to Run NFR Assessment](/docs/how-to/workflows/run-nfr-assess.md) - NFR risk assessment
|
||||
|
||||
**Use-Case Guides:**
|
||||
- [Running TEA for Enterprise](/docs/how-to/brownfield/use-tea-for-enterprise.md) - Enterprise risk management
|
||||
|
||||
## Reference
|
||||
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - `*test-design`, `*nfr-assess`, `*trace`
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Risk governance fragments
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - Risk-based testing term
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,907 @@
|
|||
---
|
||||
title: "Test Quality Standards Explained"
|
||||
description: Understanding TEA's Definition of Done for deterministic, isolated, and maintainable tests
|
||||
---
|
||||
|
||||
# Test Quality Standards Explained
|
||||
|
||||
Test quality standards define what makes a test "good" in TEA. These aren't suggestions - they're the Definition of Done that prevents tests from rotting in review.
|
||||
|
||||
## Overview
|
||||
|
||||
**TEA's Quality Principles:**
|
||||
- **Deterministic** - Same result every run
|
||||
- **Isolated** - No dependencies on other tests
|
||||
- **Explicit** - Assertions visible in test body
|
||||
- **Focused** - Single responsibility, appropriate size
|
||||
- **Fast** - Execute in reasonable time
|
||||
|
||||
**Why these matter:** Tests that violate these principles create maintenance burden, slow down development, and lose team trust.
|
||||
|
||||
## The Problem
|
||||
|
||||
### Tests That Rot in Review
|
||||
|
||||
```typescript
|
||||
// ❌ The anti-pattern: This test will rot
|
||||
test('user can do stuff', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
await page.waitForTimeout(5000); // Non-deterministic
|
||||
|
||||
if (await page.locator('.banner').isVisible()) { // Conditional
|
||||
await page.click('.dismiss');
|
||||
}
|
||||
|
||||
try { // Try-catch for flow control
|
||||
await page.click('#load-more');
|
||||
} catch (e) {
|
||||
// Silently continue
|
||||
}
|
||||
|
||||
// ... 300 more lines of test logic
|
||||
// ... no clear assertions
|
||||
});
|
||||
```
|
||||
|
||||
**What's wrong:**
|
||||
- **Hard wait** - Flaky, wastes time
|
||||
- **Conditional** - Non-deterministic behavior
|
||||
- **Try-catch** - Hides failures
|
||||
- **Too large** - Hard to maintain
|
||||
- **Vague name** - Unclear purpose
|
||||
- **No explicit assertions** - What's being tested?
|
||||
|
||||
**Result:** PR review comments: "This test is flaky, please fix" → never merged → test deleted → coverage lost
|
||||
|
||||
### AI-Generated Tests Without Standards
|
||||
|
||||
AI-generated tests without quality guardrails:
|
||||
|
||||
```typescript
|
||||
// AI generates 50 tests like this:
|
||||
test('test1', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
await page.waitForTimeout(3000);
|
||||
// ... flaky, vague, redundant
|
||||
});
|
||||
|
||||
test('test2', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
await page.waitForTimeout(3000);
|
||||
// ... duplicates test1
|
||||
});
|
||||
|
||||
// ... 48 more similar tests
|
||||
```
|
||||
|
||||
**Result:** 50 tests, 80% redundant, 90% flaky, 0% trusted by team - low-quality outputs that create maintenance burden.
|
||||
|
||||
## The Solution: TEA's Quality Standards
|
||||
|
||||
### 1. Determinism (No Flakiness)
|
||||
|
||||
**Rule:** Test produces same result every run.
|
||||
|
||||
**Requirements:**
|
||||
- ❌ No hard waits (`waitForTimeout`)
|
||||
- ❌ No conditionals for flow control (`if/else`)
|
||||
- ❌ No try-catch for flow control
|
||||
- ✅ Use network-first patterns (wait for responses)
|
||||
- ✅ Use explicit waits (waitForSelector, waitForResponse)
|
||||
|
||||
**Bad Example:**
|
||||
```typescript
|
||||
test('flaky test', async ({ page }) => {
|
||||
await page.click('button');
|
||||
await page.waitForTimeout(2000); // ❌ Might be too short
|
||||
|
||||
if (await page.locator('.modal').isVisible()) { // ❌ Non-deterministic
|
||||
await page.click('.dismiss');
|
||||
}
|
||||
|
||||
try { // ❌ Silently handles errors
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
} catch (e) {
|
||||
// Test passes even if assertion fails!
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
**Good Example (Vanilla Playwright):**
|
||||
```typescript
|
||||
test('deterministic test', async ({ page }) => {
|
||||
const responsePromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/submit') && resp.ok()
|
||||
);
|
||||
|
||||
await page.click('button');
|
||||
await responsePromise; // ✅ Wait for actual response
|
||||
|
||||
// Modal should ALWAYS show (make it deterministic)
|
||||
await expect(page.locator('.modal')).toBeVisible();
|
||||
await page.click('.dismiss');
|
||||
|
||||
// Explicit assertion (fails if not visible)
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils (Even Cleaner):**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
|
||||
test('deterministic test', async ({ page, interceptNetworkCall }) => {
|
||||
const submitCall = interceptNetworkCall({
|
||||
method: 'POST',
|
||||
url: '**/api/submit'
|
||||
});
|
||||
|
||||
await page.click('button');
|
||||
|
||||
// Wait for actual response (automatic JSON parsing)
|
||||
const { status, responseJson } = await submitCall;
|
||||
expect(status).toBe(200);
|
||||
|
||||
// Modal should ALWAYS show (make it deterministic)
|
||||
await expect(page.locator('.modal')).toBeVisible();
|
||||
await page.click('.dismiss');
|
||||
|
||||
// Explicit assertion (fails if not visible)
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Why both work:**
|
||||
- Waits for actual event (network response)
|
||||
- No conditionals (behavior is deterministic)
|
||||
- Assertions fail loudly (no silent failures)
|
||||
- Same result every run (deterministic)
|
||||
|
||||
**Playwright Utils additional benefits:**
|
||||
- Automatic JSON parsing
|
||||
- `{ status, responseJson }` structure (can validate response data)
|
||||
- No manual `await response.json()`
|
||||
|
||||
### 2. Isolation (No Dependencies)
|
||||
|
||||
**Rule:** Test runs independently, no shared state.
|
||||
|
||||
**Requirements:**
|
||||
- ✅ Self-cleaning (cleanup after test)
|
||||
- ✅ No global state dependencies
|
||||
- ✅ Can run in parallel
|
||||
- ✅ Can run in any order
|
||||
- ✅ Use unique test data
|
||||
|
||||
**Bad Example:**
|
||||
```typescript
|
||||
// ❌ Tests depend on execution order
|
||||
let userId: string; // Shared global state
|
||||
|
||||
test('create user', async ({ apiRequest }) => {
|
||||
const { body } = await apiRequest({
|
||||
method: 'POST',
|
||||
path: '/api/users',
|
||||
body: { email: 'test@example.com' } (hard-coded)
|
||||
});
|
||||
userId = body.id; // Store in global
|
||||
});
|
||||
|
||||
test('update user', async ({ apiRequest }) => {
|
||||
// Depends on previous test setting userId
|
||||
await apiRequest({
|
||||
method: 'PATCH',
|
||||
path: `/api/users/${userId}`,
|
||||
body: { name: 'Updated' }
|
||||
});
|
||||
// No cleanup - leaves user in database
|
||||
});
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Tests must run in order (can't parallelize)
|
||||
- Second test fails if first skipped (`.only`)
|
||||
- Hard-coded data causes conflicts
|
||||
- No cleanup (database fills with test data)
|
||||
|
||||
**Good Example (Vanilla Playwright):**
|
||||
```typescript
|
||||
test('should update user profile', async ({ request }) => {
|
||||
// Create unique test data
|
||||
const testEmail = `test-${Date.now()}@example.com`;
|
||||
|
||||
// Setup: Create user
|
||||
const createResp = await request.post('/api/users', {
|
||||
data: { email: testEmail, name: 'Original' }
|
||||
});
|
||||
const user = await createResp.json();
|
||||
|
||||
// Test: Update user
|
||||
const updateResp = await request.patch(`/api/users/${user.id}`, {
|
||||
data: { name: 'Updated' }
|
||||
});
|
||||
const updated = await updateResp.json();
|
||||
|
||||
expect(updated.name).toBe('Updated');
|
||||
|
||||
// Cleanup: Delete user
|
||||
await request.delete(`/api/users/${user.id}`);
|
||||
});
|
||||
```
|
||||
|
||||
**Even Better (With Playwright Utils):**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
import { faker } from '@faker-js/faker';
|
||||
|
||||
test('should update user profile', async ({ apiRequest }) => {
|
||||
// Dynamic unique test data
|
||||
const testEmail = faker.internet.email();
|
||||
|
||||
// Setup: Create user
|
||||
const { status: createStatus, body: user } = await apiRequest({
|
||||
method: 'POST',
|
||||
path: '/api/users',
|
||||
body: { email: testEmail, name: faker.person.fullName() }
|
||||
});
|
||||
|
||||
expect(createStatus).toBe(201);
|
||||
|
||||
// Test: Update user
|
||||
const { status, body: updated } = await apiRequest({
|
||||
method: 'PATCH',
|
||||
path: `/api/users/${user.id}`,
|
||||
body: { name: 'Updated Name' }
|
||||
});
|
||||
|
||||
expect(status).toBe(200);
|
||||
expect(updated.name).toBe('Updated Name');
|
||||
|
||||
// Cleanup: Delete user
|
||||
await apiRequest({
|
||||
method: 'DELETE',
|
||||
path: `/api/users/${user.id}`
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Playwright Utils Benefits:**
|
||||
- `{ status, body }` destructuring (cleaner than `response.status()` + `await response.json()`)
|
||||
- No manual `await response.json()`
|
||||
- Automatic retry for 5xx errors
|
||||
- Optional schema validation with `.validateSchema()`
|
||||
|
||||
**Why it works:**
|
||||
- No global state
|
||||
- Unique test data (no conflicts)
|
||||
- Self-cleaning (deletes user)
|
||||
- Can run in parallel
|
||||
- Can run in any order
|
||||
|
||||
### 3. Explicit Assertions (No Hidden Validation)
|
||||
|
||||
**Rule:** Assertions visible in test body, not abstracted.
|
||||
|
||||
**Requirements:**
|
||||
- ✅ Assertions in test code (not helper functions)
|
||||
- ✅ Specific assertions (not generic `toBeTruthy`)
|
||||
- ✅ Meaningful expectations (test actual behavior)
|
||||
|
||||
**Bad Example:**
|
||||
```typescript
|
||||
// ❌ Assertions hidden in helper
|
||||
async function verifyProfilePage(page: Page) {
|
||||
// Assertions buried in helper (not visible in test)
|
||||
await expect(page.locator('h1')).toBeVisible();
|
||||
await expect(page.locator('.email')).toContainText('@');
|
||||
await expect(page.locator('.name')).not.toBeEmpty();
|
||||
}
|
||||
|
||||
test('profile page', async ({ page }) => {
|
||||
await page.goto('/profile');
|
||||
await verifyProfilePage(page); // What's being verified?
|
||||
});
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Can't see what's tested (need to read helper)
|
||||
- Hard to debug failures (which assertion failed?)
|
||||
- Reduces test readability
|
||||
- Hides important validation
|
||||
|
||||
**Good Example:**
|
||||
```typescript
|
||||
// ✅ Assertions explicit in test
|
||||
test('should display profile with correct data', async ({ page }) => {
|
||||
await page.goto('/profile');
|
||||
|
||||
// Explicit assertions - clear what's tested
|
||||
await expect(page.locator('h1')).toContainText('Test User');
|
||||
await expect(page.locator('.email')).toContainText('test@example.com');
|
||||
await expect(page.locator('.bio')).toContainText('Software Engineer');
|
||||
await expect(page.locator('img[alt="Avatar"]')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Why it works:**
|
||||
- See what's tested at a glance
|
||||
- Debug failures easily (know which assertion failed)
|
||||
- Test is self-documenting
|
||||
- No hidden behavior
|
||||
|
||||
**Exception:** Use helper for setup/cleanup, not assertions.
|
||||
|
||||
### 4. Focused Tests (Appropriate Size)
|
||||
|
||||
**Rule:** Test has single responsibility, reasonable size.
|
||||
|
||||
**Requirements:**
|
||||
- ✅ Test size < 300 lines
|
||||
- ✅ Single responsibility (test one thing well)
|
||||
- ✅ Clear describe/test names
|
||||
- ✅ Appropriate scope (not too granular, not too broad)
|
||||
|
||||
**Bad Example:**
|
||||
```typescript
|
||||
// ❌ 500-line test testing everything
|
||||
test('complete user flow', async ({ page }) => {
|
||||
// Registration (50 lines)
|
||||
await page.goto('/register');
|
||||
await page.fill('#email', 'test@example.com');
|
||||
// ... 48 more lines
|
||||
|
||||
// Profile setup (100 lines)
|
||||
await page.goto('/profile');
|
||||
// ... 98 more lines
|
||||
|
||||
// Settings configuration (150 lines)
|
||||
await page.goto('/settings');
|
||||
// ... 148 more lines
|
||||
|
||||
// Data export (200 lines)
|
||||
await page.goto('/export');
|
||||
// ... 198 more lines
|
||||
|
||||
// Total: 500 lines, testing 4 different features
|
||||
});
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Failure in line 50 prevents testing lines 51-500
|
||||
- Hard to understand (what's being tested?)
|
||||
- Slow to execute (testing too much)
|
||||
- Hard to debug (which feature failed?)
|
||||
|
||||
**Good Example:**
|
||||
```typescript
|
||||
// ✅ Focused tests - one responsibility each
|
||||
|
||||
test('should register new user', async ({ page }) => {
|
||||
await page.goto('/register');
|
||||
await page.fill('#email', 'test@example.com');
|
||||
await page.fill('#password', 'password123');
|
||||
await page.click('button[type="submit"]');
|
||||
|
||||
await expect(page).toHaveURL('/welcome');
|
||||
await expect(page.locator('h1')).toContainText('Welcome');
|
||||
});
|
||||
|
||||
test('should configure user profile', async ({ page, authSession }) => {
|
||||
await authSession.login({ email: 'test@example.com', password: 'pass' });
|
||||
await page.goto('/profile');
|
||||
|
||||
await page.fill('#name', 'Test User');
|
||||
await page.fill('#bio', 'Software Engineer');
|
||||
await page.click('button:has-text("Save")');
|
||||
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
});
|
||||
|
||||
// ... separate tests for settings, export (each < 50 lines)
|
||||
```
|
||||
|
||||
**Why it works:**
|
||||
- Each test has one responsibility
|
||||
- Failure is easy to diagnose
|
||||
- Can run tests independently
|
||||
- Test names describe exactly what's tested
|
||||
|
||||
### 5. Fast Execution (Performance Budget)
|
||||
|
||||
**Rule:** Individual test executes in < 1.5 minutes.
|
||||
|
||||
**Requirements:**
|
||||
- ✅ Test execution < 90 seconds
|
||||
- ✅ Efficient selectors (getByRole > XPath)
|
||||
- ✅ Minimal redundant actions
|
||||
- ✅ Parallel execution enabled
|
||||
|
||||
**Bad Example:**
|
||||
```typescript
|
||||
// ❌ Slow test (3+ minutes)
|
||||
test('slow test', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
await page.waitForTimeout(10000); // 10s wasted
|
||||
|
||||
// Navigate through 10 pages (2 minutes)
|
||||
for (let i = 1; i <= 10; i++) {
|
||||
await page.click(`a[href="/page-${i}"]`);
|
||||
await page.waitForTimeout(5000); // 5s per page = 50s wasted
|
||||
}
|
||||
|
||||
// Complex XPath selector (slow)
|
||||
await page.locator('//div[@class="container"]/section[3]/div[2]/p').click();
|
||||
|
||||
// More waiting
|
||||
await page.waitForTimeout(30000); // 30s wasted
|
||||
|
||||
await expect(page.locator('.result')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Total time:** 3+ minutes (95 seconds wasted on hard waits)
|
||||
|
||||
**Good Example (Vanilla Playwright):**
|
||||
```typescript
|
||||
// ✅ Fast test (< 10 seconds)
|
||||
test('fast test', async ({ page }) => {
|
||||
// Set up response wait
|
||||
const apiPromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/result') && resp.ok()
|
||||
);
|
||||
|
||||
await page.goto('/');
|
||||
|
||||
// Direct navigation (skip intermediate pages)
|
||||
await page.goto('/page-10');
|
||||
|
||||
// Efficient selector
|
||||
await page.getByRole('button', { name: 'Submit' }).click();
|
||||
|
||||
// Wait for actual response (fast when API is fast)
|
||||
await apiPromise;
|
||||
|
||||
await expect(page.locator('.result')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
|
||||
test('fast test', async ({ page, interceptNetworkCall }) => {
|
||||
// Set up interception
|
||||
const resultCall = interceptNetworkCall({
|
||||
method: 'GET',
|
||||
url: '**/api/result'
|
||||
});
|
||||
|
||||
await page.goto('/');
|
||||
|
||||
// Direct navigation (skip intermediate pages)
|
||||
await page.goto('/page-10');
|
||||
|
||||
// Efficient selector
|
||||
await page.getByRole('button', { name: 'Submit' }).click();
|
||||
|
||||
// Wait for actual response (automatic JSON parsing)
|
||||
const { status, responseJson } = await resultCall;
|
||||
|
||||
expect(status).toBe(200);
|
||||
await expect(page.locator('.result')).toBeVisible();
|
||||
|
||||
// Can also validate response data if needed
|
||||
// expect(responseJson.data).toBeDefined();
|
||||
});
|
||||
```
|
||||
|
||||
**Total time:** < 10 seconds (no wasted waits)
|
||||
|
||||
**Both examples achieve:**
|
||||
- No hard waits (wait for actual events)
|
||||
- Direct navigation (skip unnecessary steps)
|
||||
- Efficient selectors (getByRole)
|
||||
- Fast execution
|
||||
|
||||
**Playwright Utils bonus:**
|
||||
- Can validate API response data easily
|
||||
- Automatic JSON parsing
|
||||
- Cleaner API
|
||||
|
||||
## TEA's Quality Scoring
|
||||
|
||||
TEA reviews tests against these standards in `*test-review`:
|
||||
|
||||
### Scoring Categories (100 points total)
|
||||
|
||||
**Determinism (35 points):**
|
||||
- No hard waits: 10 points
|
||||
- No conditionals: 10 points
|
||||
- No try-catch flow: 10 points
|
||||
- Network-first patterns: 5 points
|
||||
|
||||
**Isolation (25 points):**
|
||||
- Self-cleaning: 15 points
|
||||
- No global state: 5 points
|
||||
- Parallel-safe: 5 points
|
||||
|
||||
**Assertions (20 points):**
|
||||
- Explicit in test body: 10 points
|
||||
- Specific and meaningful: 10 points
|
||||
|
||||
**Structure (10 points):**
|
||||
- Test size < 300 lines: 5 points
|
||||
- Clear naming: 5 points
|
||||
|
||||
**Performance (10 points):**
|
||||
- Execution time < 1.5 min: 10 points
|
||||
|
||||
#### Quality Scoring Breakdown
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'14px'}}}%%
|
||||
pie title Test Quality Score (100 points)
|
||||
"Determinism" : 35
|
||||
"Isolation" : 25
|
||||
"Assertions" : 20
|
||||
"Structure" : 10
|
||||
"Performance" : 10
|
||||
```
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'13px'}}}%%
|
||||
flowchart LR
|
||||
subgraph Det[Determinism - 35 pts]
|
||||
D1[No hard waits<br/>10 pts]
|
||||
D2[No conditionals<br/>10 pts]
|
||||
D3[No try-catch flow<br/>10 pts]
|
||||
D4[Network-first<br/>5 pts]
|
||||
end
|
||||
|
||||
subgraph Iso[Isolation - 25 pts]
|
||||
I1[Self-cleaning<br/>15 pts]
|
||||
I2[No global state<br/>5 pts]
|
||||
I3[Parallel-safe<br/>5 pts]
|
||||
end
|
||||
|
||||
subgraph Assrt[Assertions - 20 pts]
|
||||
A1[Explicit in body<br/>10 pts]
|
||||
A2[Specific/meaningful<br/>10 pts]
|
||||
end
|
||||
|
||||
subgraph Struct[Structure - 10 pts]
|
||||
S1[Size < 300 lines<br/>5 pts]
|
||||
S2[Clear naming<br/>5 pts]
|
||||
end
|
||||
|
||||
subgraph Perf[Performance - 10 pts]
|
||||
P1[Time < 1.5 min<br/>10 pts]
|
||||
end
|
||||
|
||||
Det --> Total([Total: 100 points])
|
||||
Iso --> Total
|
||||
Assrt --> Total
|
||||
Struct --> Total
|
||||
Perf --> Total
|
||||
|
||||
style Det fill:#ffebee,stroke:#c62828,stroke-width:2px
|
||||
style Iso fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
|
||||
style Assrt fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px
|
||||
style Struct fill:#fff9c4,stroke:#f57f17,stroke-width:2px
|
||||
style Perf fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px
|
||||
style Total fill:#fff,stroke:#000,stroke-width:3px
|
||||
```
|
||||
|
||||
### Score Interpretation
|
||||
|
||||
| Score | Interpretation | Action |
|
||||
| ---------- | -------------- | -------------------------------------- |
|
||||
| **90-100** | Excellent | Production-ready, minimal changes |
|
||||
| **80-89** | Good | Minor improvements recommended |
|
||||
| **70-79** | Acceptable | Address recommendations before release |
|
||||
| **60-69** | Needs Work | Fix critical issues |
|
||||
| **< 60** | Critical | Significant refactoring needed |
|
||||
|
||||
## Comparison: Good vs Bad Tests
|
||||
|
||||
### Example: User Login
|
||||
|
||||
**Bad Test (Score: 45/100):**
|
||||
```typescript
|
||||
test('login test', async ({ page }) => { // Vague name
|
||||
await page.goto('/login');
|
||||
await page.waitForTimeout(3000); // -10 (hard wait)
|
||||
|
||||
await page.fill('[name="email"]', 'test@example.com');
|
||||
await page.fill('[name="password"]', 'password');
|
||||
|
||||
if (await page.locator('.remember-me').isVisible()) { // -10 (conditional)
|
||||
await page.click('.remember-me');
|
||||
}
|
||||
|
||||
await page.click('button');
|
||||
|
||||
try { // -10 (try-catch flow)
|
||||
await page.waitForURL('/dashboard', { timeout: 5000 });
|
||||
} catch (e) {
|
||||
// Ignore navigation failure
|
||||
}
|
||||
|
||||
// No assertions! -10
|
||||
// No cleanup! -10
|
||||
});
|
||||
```
|
||||
|
||||
**Issues:**
|
||||
- Determinism: 5/35 (hard wait, conditional, try-catch)
|
||||
- Isolation: 10/25 (no cleanup)
|
||||
- Assertions: 0/20 (no assertions!)
|
||||
- Structure: 15/10 (okay)
|
||||
- Performance: 5/10 (slow)
|
||||
- **Total: 45/100**
|
||||
|
||||
**Good Test (Score: 95/100):**
|
||||
```typescript
|
||||
test('should login with valid credentials and redirect to dashboard', async ({ page, authSession }) => {
|
||||
// Use fixture for deterministic auth
|
||||
const loginPromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/auth/login') && resp.ok()
|
||||
);
|
||||
|
||||
await page.goto('/login');
|
||||
await page.getByLabel('Email').fill('test@example.com');
|
||||
await page.getByLabel('Password').fill('password123');
|
||||
await page.getByRole('button', { name: 'Sign in' }).click();
|
||||
|
||||
// Wait for actual API response
|
||||
const response = await loginPromise;
|
||||
const { token } = await response.json();
|
||||
|
||||
// Explicit assertions
|
||||
expect(token).toBeDefined();
|
||||
await expect(page).toHaveURL('/dashboard');
|
||||
await expect(page.getByText('Welcome back')).toBeVisible();
|
||||
|
||||
// Cleanup handled by authSession fixture
|
||||
});
|
||||
```
|
||||
|
||||
**Quality:**
|
||||
- Determinism: 35/35 (network-first, no conditionals)
|
||||
- Isolation: 25/25 (fixture handles cleanup)
|
||||
- Assertions: 20/20 (explicit and specific)
|
||||
- Structure: 10/10 (clear name, focused)
|
||||
- Performance: 5/10 (< 1 min)
|
||||
- **Total: 95/100**
|
||||
|
||||
### Example: API Testing
|
||||
|
||||
**Bad Test (Score: 50/100):**
|
||||
```typescript
|
||||
test('api test', async ({ request }) => {
|
||||
const response = await request.post('/api/users', {
|
||||
data: { email: 'test@example.com' } // Hard-coded (conflicts)
|
||||
});
|
||||
|
||||
if (response.ok()) { // Conditional
|
||||
const user = await response.json();
|
||||
// Weak assertion
|
||||
expect(user).toBeTruthy();
|
||||
}
|
||||
|
||||
// No cleanup - user left in database
|
||||
});
|
||||
```
|
||||
|
||||
**Good Test (Score: 92/100):**
|
||||
```typescript
|
||||
test('should create user with valid data', async ({ apiRequest }) => {
|
||||
// Unique test data
|
||||
const testEmail = `test-${Date.now()}@example.com`;
|
||||
|
||||
// Create user
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'POST',
|
||||
path: '/api/users',
|
||||
body: { email: testEmail, name: 'Test User' }
|
||||
});
|
||||
|
||||
// Explicit assertions
|
||||
expect(status).toBe(201);
|
||||
expect(body.id).toBeDefined();
|
||||
expect(body.email).toBe(testEmail);
|
||||
expect(body.name).toBe('Test User');
|
||||
|
||||
// Cleanup
|
||||
await apiRequest({
|
||||
method: 'DELETE',
|
||||
path: `/api/users/${body.id}`
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## How TEA Enforces Standards
|
||||
|
||||
### During Test Generation (`*atdd`, `*automate`)
|
||||
|
||||
TEA generates tests following standards by default:
|
||||
|
||||
```typescript
|
||||
// TEA-generated test (automatically follows standards)
|
||||
test('should submit contact form', async ({ page }) => {
|
||||
// Network-first pattern (no hard waits)
|
||||
const submitPromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/contact') && resp.ok()
|
||||
);
|
||||
|
||||
// Accessible selectors (resilient)
|
||||
await page.getByLabel('Name').fill('Test User');
|
||||
await page.getByLabel('Email').fill('test@example.com');
|
||||
await page.getByLabel('Message').fill('Test message');
|
||||
await page.getByRole('button', { name: 'Send' }).click();
|
||||
|
||||
const response = await submitPromise;
|
||||
const result = await response.json();
|
||||
|
||||
// Explicit assertions
|
||||
expect(result.success).toBe(true);
|
||||
await expect(page.getByText('Message sent')).toBeVisible();
|
||||
|
||||
// Size: 15 lines (< 300 ✓)
|
||||
// Execution: ~2 seconds (< 90s ✓)
|
||||
});
|
||||
```
|
||||
|
||||
### During Test Review (*test-review)
|
||||
|
||||
TEA audits tests and flags violations:
|
||||
|
||||
```markdown
|
||||
## Critical Issues
|
||||
|
||||
### Hard Wait Detected (tests/login.spec.ts:23)
|
||||
**Issue:** `await page.waitForTimeout(3000)`
|
||||
**Score Impact:** -10 (Determinism)
|
||||
**Fix:** Use network-first pattern
|
||||
|
||||
### Conditional Flow Control (tests/profile.spec.ts:45)
|
||||
**Issue:** `if (await page.locator('.banner').isVisible())`
|
||||
**Score Impact:** -10 (Determinism)
|
||||
**Fix:** Make banner presence deterministic
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Extract Fixture (tests/auth.spec.ts)
|
||||
**Issue:** Login code repeated 5 times
|
||||
**Score Impact:** -3 (Structure)
|
||||
**Fix:** Extract to authSession fixture
|
||||
```
|
||||
|
||||
## Definition of Done Checklist
|
||||
|
||||
When is a test "done"?
|
||||
|
||||
**Test Quality DoD:**
|
||||
- [ ] No hard waits (`waitForTimeout`)
|
||||
- [ ] No conditionals for flow control
|
||||
- [ ] No try-catch for flow control
|
||||
- [ ] Network-first patterns used
|
||||
- [ ] Assertions explicit in test body
|
||||
- [ ] Test size < 300 lines
|
||||
- [ ] Clear, descriptive test name
|
||||
- [ ] Self-cleaning (cleanup in afterEach or test)
|
||||
- [ ] Unique test data (no hard-coded values)
|
||||
- [ ] Execution time < 1.5 minutes
|
||||
- [ ] Can run in parallel
|
||||
- [ ] Can run in any order
|
||||
|
||||
**Code Review DoD:**
|
||||
- [ ] Test quality score > 80
|
||||
- [ ] No critical issues from `*test-review`
|
||||
- [ ] Follows project patterns (fixtures, selectors)
|
||||
- [ ] Test reviewed by team member
|
||||
|
||||
## Common Quality Issues
|
||||
|
||||
### Issue: "My test needs conditionals for optional elements"
|
||||
|
||||
**Wrong approach:**
|
||||
```typescript
|
||||
if (await page.locator('.banner').isVisible()) {
|
||||
await page.click('.dismiss');
|
||||
}
|
||||
```
|
||||
|
||||
**Right approach - Make it deterministic:**
|
||||
```typescript
|
||||
// Option 1: Always expect banner
|
||||
await expect(page.locator('.banner')).toBeVisible();
|
||||
await page.click('.dismiss');
|
||||
|
||||
// Option 2: Test both scenarios separately
|
||||
test('should show banner for new users', ...);
|
||||
test('should not show banner for returning users', ...);
|
||||
```
|
||||
|
||||
### Issue: "My test needs try-catch for error handling"
|
||||
|
||||
**Wrong approach:**
|
||||
```typescript
|
||||
try {
|
||||
await page.click('#optional-button');
|
||||
} catch (e) {
|
||||
// Silently continue
|
||||
}
|
||||
```
|
||||
|
||||
**Right approach - Make failures explicit:**
|
||||
```typescript
|
||||
// Option 1: Button should exist
|
||||
await page.click('#optional-button'); // Fails loudly if missing
|
||||
|
||||
// Option 2: Button might not exist (test both)
|
||||
test('should work with optional button', async ({ page }) => {
|
||||
const hasButton = await page.locator('#optional-button').count() > 0;
|
||||
if (hasButton) {
|
||||
await page.click('#optional-button');
|
||||
}
|
||||
// But now you're testing optional behavior explicitly
|
||||
});
|
||||
```
|
||||
|
||||
### Issue: "Hard waits are easier than network patterns"
|
||||
|
||||
**Short-term:** Hard waits seem simpler
|
||||
**Long-term:** Flaky tests waste more time than learning network patterns
|
||||
|
||||
**Investment:**
|
||||
- 30 minutes to learn network-first patterns
|
||||
- Prevents hundreds of hours debugging flaky tests
|
||||
- Tests run faster (no wasted waits)
|
||||
- Team trusts test suite
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
For detailed test quality patterns, see:
|
||||
- [Test Quality Fragment](/docs/reference/tea/knowledge-base.md#quality-standards)
|
||||
- [Test Levels Framework Fragment](/docs/reference/tea/knowledge-base.md#quality-standards)
|
||||
- [Complete Knowledge Base Index](/docs/reference/tea/knowledge-base.md)
|
||||
|
||||
## Related Concepts
|
||||
|
||||
**Core TEA Concepts:**
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Quality scales with risk
|
||||
- [Knowledge Base System](/docs/explanation/tea/knowledge-base-system.md) - How standards are enforced
|
||||
- [Engagement Models](/docs/explanation/tea/engagement-models.md) - Quality in different models
|
||||
|
||||
**Technical Patterns:**
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Determinism explained
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Isolation through fixtures
|
||||
|
||||
**Overview:**
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Quality standards in lifecycle
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - Why quality matters
|
||||
|
||||
## Practical Guides
|
||||
|
||||
**Workflow Guides:**
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md) - Audit against these standards
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Generate quality tests
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Expand with quality
|
||||
|
||||
**Use-Case Guides:**
|
||||
- [Using TEA with Existing Tests](/docs/how-to/brownfield/use-tea-with-existing-tests.md) - Improve legacy quality
|
||||
- [Running TEA for Enterprise](/docs/how-to/brownfield/use-tea-for-enterprise.md) - Enterprise quality thresholds
|
||||
|
||||
## Reference
|
||||
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - *test-review command
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Test quality fragment
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - TEA terminology
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,526 @@
|
|||
---
|
||||
title: "Running TEA for Enterprise Projects"
|
||||
description: Use TEA with compliance, security, and regulatory requirements in enterprise environments
|
||||
---
|
||||
|
||||
# Running TEA for Enterprise Projects
|
||||
|
||||
Use TEA on enterprise projects with compliance, security, audit, and regulatory requirements. This guide covers NFR assessment, audit trails, and evidence collection.
|
||||
|
||||
## When to Use This
|
||||
|
||||
- Enterprise track projects (not Quick Flow or simple BMad Method)
|
||||
- Compliance requirements (SOC 2, HIPAA, GDPR, etc.)
|
||||
- Security-critical applications (finance, healthcare, government)
|
||||
- Audit trail requirements
|
||||
- Strict NFR thresholds (performance, security, reliability)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed (Enterprise track selected)
|
||||
- TEA agent available
|
||||
- Compliance requirements documented
|
||||
- Stakeholders identified (who approves gates)
|
||||
|
||||
## Enterprise-Specific TEA Workflows
|
||||
|
||||
### NFR Assessment (*nfr-assess)
|
||||
|
||||
**Purpose:** Validate non-functional requirements with evidence.
|
||||
|
||||
**When:** Phase 2 (early) and Release Gate
|
||||
|
||||
**Why Enterprise Needs This:**
|
||||
- Compliance mandates specific thresholds
|
||||
- Audit trails required for certification
|
||||
- Security requirements are non-negotiable
|
||||
- Performance SLAs are contractual
|
||||
|
||||
**Example:**
|
||||
```
|
||||
*nfr-assess
|
||||
|
||||
Categories: Security, Performance, Reliability, Maintainability
|
||||
|
||||
Security thresholds:
|
||||
- Zero critical vulnerabilities (required by SOC 2)
|
||||
- All endpoints require authentication
|
||||
- Data encrypted at rest (FIPS 140-2)
|
||||
- Audit logging on all data access
|
||||
|
||||
Evidence:
|
||||
- Security scan: reports/nessus-scan.pdf
|
||||
- Penetration test: reports/pentest-2026-01.pdf
|
||||
- Compliance audit: reports/soc2-evidence.zip
|
||||
```
|
||||
|
||||
**Output:** NFR assessment with PASS/CONCERNS/FAIL for each category.
|
||||
|
||||
### Trace with Audit Evidence (*trace)
|
||||
|
||||
**Purpose:** Requirements traceability with audit trail.
|
||||
|
||||
**When:** Phase 2 (baseline), Phase 4 (refresh), Release Gate
|
||||
|
||||
**Why Enterprise Needs This:**
|
||||
- Auditors require requirements-to-test mapping
|
||||
- Compliance certifications need traceability
|
||||
- Regulatory bodies want evidence
|
||||
|
||||
**Example:**
|
||||
```
|
||||
*trace Phase 1
|
||||
|
||||
Requirements: PRD.md (with compliance requirements)
|
||||
Test location: tests/
|
||||
|
||||
Output: traceability-matrix.md with:
|
||||
- Requirement-to-test mapping
|
||||
- Compliance requirement coverage
|
||||
- Gap prioritization
|
||||
- Recommendations
|
||||
```
|
||||
|
||||
**For Release Gate:**
|
||||
```
|
||||
*trace Phase 2
|
||||
|
||||
Generate gate-decision-{gate_type}-{story_id}.md with:
|
||||
- Evidence references
|
||||
- Approver signatures
|
||||
- Compliance checklist
|
||||
- Decision rationale
|
||||
```
|
||||
|
||||
### Test Design with Compliance Focus (*test-design)
|
||||
|
||||
**Purpose:** Risk assessment with compliance and security focus.
|
||||
|
||||
**When:** Phase 3 (system-level), Phase 4 (epic-level)
|
||||
|
||||
**Why Enterprise Needs This:**
|
||||
- Security architecture alignment required
|
||||
- Compliance requirements must be testable
|
||||
- Performance requirements are contractual
|
||||
|
||||
**Example:**
|
||||
```
|
||||
*test-design
|
||||
|
||||
Mode: System-level
|
||||
|
||||
Focus areas:
|
||||
- Security architecture (authentication, authorization, encryption)
|
||||
- Performance requirements (SLA: P99 <200ms)
|
||||
- Compliance (HIPAA PHI handling, audit logging)
|
||||
|
||||
Output: test-design-system.md with:
|
||||
- Security testing strategy
|
||||
- Compliance requirement → test mapping
|
||||
- Performance testing plan
|
||||
- Audit logging validation
|
||||
```
|
||||
|
||||
## Enterprise TEA Lifecycle
|
||||
|
||||
### Phase 1: Discovery (Optional but Recommended)
|
||||
|
||||
**Research compliance requirements:**
|
||||
```
|
||||
Analyst: *research
|
||||
|
||||
Topics:
|
||||
- Industry compliance (SOC 2, HIPAA, GDPR)
|
||||
- Security standards (OWASP Top 10)
|
||||
- Performance benchmarks (industry P99)
|
||||
```
|
||||
|
||||
### Phase 2: Planning (Required)
|
||||
|
||||
**1. Define NFRs early:**
|
||||
```
|
||||
PM: *prd
|
||||
|
||||
Include in PRD:
|
||||
- Security requirements (authentication, encryption)
|
||||
- Performance SLAs (response time, throughput)
|
||||
- Reliability targets (uptime, RTO, RPO)
|
||||
- Compliance mandates (data retention, audit logs)
|
||||
```
|
||||
|
||||
**2. Assess NFRs:**
|
||||
```
|
||||
TEA: *nfr-assess
|
||||
|
||||
Categories: All (Security, Performance, Reliability, Maintainability)
|
||||
|
||||
Output: nfr-assessment.md
|
||||
- NFR requirements documented
|
||||
- Acceptance criteria defined
|
||||
- Test strategy planned
|
||||
```
|
||||
|
||||
**3. Baseline (brownfield only):**
|
||||
```
|
||||
TEA: *trace Phase 1
|
||||
|
||||
Establish baseline coverage before new work
|
||||
```
|
||||
|
||||
### Phase 3: Solutioning (Required)
|
||||
|
||||
**1. Architecture with testability review:**
|
||||
```
|
||||
Architect: *architecture
|
||||
|
||||
TEA: *test-design (system-level)
|
||||
|
||||
Focus:
|
||||
- Security architecture testability
|
||||
- Performance testing strategy
|
||||
- Compliance requirement mapping
|
||||
```
|
||||
|
||||
**2. Test infrastructure:**
|
||||
```
|
||||
TEA: *framework
|
||||
|
||||
Requirements:
|
||||
- Separate test environments (dev, staging, prod-mirror)
|
||||
- Secure test data handling (PHI, PII)
|
||||
- Audit logging in tests
|
||||
```
|
||||
|
||||
**3. CI/CD with compliance:**
|
||||
```
|
||||
TEA: *ci
|
||||
|
||||
Requirements:
|
||||
- Secrets management (Vault, AWS Secrets Manager)
|
||||
- Test isolation (no cross-contamination)
|
||||
- Artifact retention (compliance audit trail)
|
||||
- Access controls (who can run production tests)
|
||||
```
|
||||
|
||||
### Phase 4: Implementation (Required)
|
||||
|
||||
**Per epic:**
|
||||
```
|
||||
1. TEA: *test-design (epic-level)
|
||||
Focus: Compliance, security, performance for THIS epic
|
||||
|
||||
2. TEA: *atdd (optional)
|
||||
Generate tests including security/compliance scenarios
|
||||
|
||||
3. DEV: Implement story
|
||||
|
||||
4. TEA: *automate
|
||||
Expand coverage including compliance edge cases
|
||||
|
||||
5. TEA: *test-review
|
||||
Audit quality (score >80 per epic, rises to >85 at release)
|
||||
|
||||
6. TEA: *trace Phase 1
|
||||
Refresh coverage, verify compliance requirements tested
|
||||
```
|
||||
|
||||
### Release Gate (Required)
|
||||
|
||||
**1. Final NFR assessment:**
|
||||
```
|
||||
TEA: *nfr-assess
|
||||
|
||||
All categories (if not done earlier)
|
||||
Latest evidence (performance tests, security scans)
|
||||
```
|
||||
|
||||
**2. Final quality audit:**
|
||||
```
|
||||
TEA: *test-review tests/
|
||||
|
||||
Full suite review
|
||||
Quality target: >85 for enterprise
|
||||
```
|
||||
|
||||
**3. Gate decision:**
|
||||
```
|
||||
TEA: *trace Phase 2
|
||||
|
||||
Evidence required:
|
||||
- traceability-matrix.md (from Phase 1)
|
||||
- test-review.md (from quality audit)
|
||||
- nfr-assessment.md (from NFR assessment)
|
||||
- Test execution results (must have test results available)
|
||||
|
||||
Decision: PASS/CONCERNS/FAIL/WAIVED
|
||||
|
||||
Archive all artifacts for compliance audit
|
||||
```
|
||||
|
||||
**Note:** Phase 2 requires test execution results. If results aren't available, Phase 2 will be skipped.
|
||||
|
||||
**4. Archive for audit:**
|
||||
```
|
||||
Archive:
|
||||
- All test results
|
||||
- Coverage reports
|
||||
- NFR assessments
|
||||
- Gate decisions
|
||||
- Approver signatures
|
||||
|
||||
Retention: Per compliance requirements (7 years for HIPAA)
|
||||
```
|
||||
|
||||
## Enterprise-Specific Requirements
|
||||
|
||||
### Evidence Collection
|
||||
|
||||
**Required artifacts:**
|
||||
- Requirements traceability matrix
|
||||
- Test execution results (with timestamps)
|
||||
- NFR assessment reports
|
||||
- Security scan results
|
||||
- Performance test results
|
||||
- Gate decision records
|
||||
- Approver signatures
|
||||
|
||||
**Storage:**
|
||||
```
|
||||
compliance/
|
||||
├── 2026-Q1/
|
||||
│ ├── release-1.2.0/
|
||||
│ │ ├── traceability-matrix.md
|
||||
│ │ ├── test-review.md
|
||||
│ │ ├── nfr-assessment.md
|
||||
│ │ ├── gate-decision-release-v1.2.0.md
|
||||
│ │ ├── test-results/
|
||||
│ │ ├── security-scans/
|
||||
│ │ └── approvals.pdf
|
||||
```
|
||||
|
||||
**Retention:** 7 years (HIPAA), 3 years (SOC 2), per your compliance needs
|
||||
|
||||
### Approver Workflows
|
||||
|
||||
**Multi-level approval required:**
|
||||
|
||||
```markdown
|
||||
## Gate Approvals Required
|
||||
|
||||
### Technical Approval
|
||||
- [ ] QA Lead - Test coverage adequate
|
||||
- [ ] Tech Lead - Technical quality acceptable
|
||||
- [ ] Security Lead - Security requirements met
|
||||
|
||||
### Business Approval
|
||||
- [ ] Product Manager - Business requirements met
|
||||
- [ ] Compliance Officer - Regulatory requirements met
|
||||
|
||||
### Executive Approval (for major releases)
|
||||
- [ ] VP Engineering - Overall quality acceptable
|
||||
- [ ] CTO - Architecture approved for production
|
||||
```
|
||||
|
||||
### Compliance Checklists
|
||||
|
||||
**SOC 2 Example:**
|
||||
```markdown
|
||||
## SOC 2 Compliance Checklist
|
||||
|
||||
### Access Controls
|
||||
- [ ] All API endpoints require authentication
|
||||
- [ ] Authorization tested for all protected resources
|
||||
- [ ] Session management secure (token expiration tested)
|
||||
|
||||
### Audit Logging
|
||||
- [ ] All data access logged
|
||||
- [ ] Logs immutable (append-only)
|
||||
- [ ] Log retention policy enforced
|
||||
|
||||
### Data Protection
|
||||
- [ ] Data encrypted at rest (tested)
|
||||
- [ ] Data encrypted in transit (HTTPS enforced)
|
||||
- [ ] PII handling compliant (masking tested)
|
||||
|
||||
### Testing Evidence
|
||||
- [ ] Test coverage >80% (verified)
|
||||
- [ ] Security tests passing (100%)
|
||||
- [ ] Traceability matrix complete
|
||||
```
|
||||
|
||||
**HIPAA Example:**
|
||||
```markdown
|
||||
## HIPAA Compliance Checklist
|
||||
|
||||
### PHI Protection
|
||||
- [ ] PHI encrypted at rest (AES-256)
|
||||
- [ ] PHI encrypted in transit (TLS 1.3)
|
||||
- [ ] PHI access logged (audit trail)
|
||||
|
||||
### Access Controls
|
||||
- [ ] Role-based access control (RBAC tested)
|
||||
- [ ] Minimum necessary access (tested)
|
||||
- [ ] Authentication strong (MFA tested)
|
||||
|
||||
### Breach Notification
|
||||
- [ ] Breach detection tested
|
||||
- [ ] Notification workflow tested
|
||||
- [ ] Incident response plan tested
|
||||
```
|
||||
|
||||
## Enterprise Tips
|
||||
|
||||
### Start with Security
|
||||
|
||||
**Priority 1:** Security requirements
|
||||
```
|
||||
1. Document all security requirements
|
||||
2. Generate security tests with *atdd
|
||||
3. Run security test suite
|
||||
4. Pass security audit BEFORE moving forward
|
||||
```
|
||||
|
||||
**Why:** Security failures block everything in enterprise.
|
||||
|
||||
**Example: RBAC Testing**
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
test('should enforce role-based access', async ({ request }) => {
|
||||
// Login as regular user
|
||||
const userResp = await request.post('/api/auth/login', {
|
||||
data: { email: 'user@example.com', password: 'pass' }
|
||||
});
|
||||
const { token: userToken } = await userResp.json();
|
||||
|
||||
// Try to access admin endpoint
|
||||
const adminResp = await request.get('/api/admin/users', {
|
||||
headers: { Authorization: `Bearer ${userToken}` }
|
||||
});
|
||||
|
||||
expect(adminResp.status()).toBe(403); // Forbidden
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils (Cleaner, Reusable):**
|
||||
```typescript
|
||||
import { test as base, expect } from '@playwright/test';
|
||||
import { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { createAuthFixtures } from '@seontechnologies/playwright-utils/auth-session';
|
||||
import { mergeTests } from '@playwright/test';
|
||||
|
||||
const authFixtureTest = base.extend(createAuthFixtures());
|
||||
export const testWithAuth = mergeTests(apiRequestFixture, authFixtureTest);
|
||||
|
||||
testWithAuth('should enforce role-based access', async ({ apiRequest, authToken }) => {
|
||||
// Auth token from fixture (configured for 'user' role)
|
||||
const { status } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/admin/users', // Admin endpoint
|
||||
headers: { Authorization: `Bearer ${authToken}` }
|
||||
});
|
||||
|
||||
expect(status).toBe(403); // Regular user denied
|
||||
});
|
||||
|
||||
testWithAuth('admin can access admin endpoint', async ({ apiRequest, authToken, authOptions }) => {
|
||||
// Override to admin role
|
||||
authOptions.userIdentifier = 'admin';
|
||||
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/admin/users',
|
||||
headers: { Authorization: `Bearer ${authToken}` }
|
||||
});
|
||||
|
||||
expect(status).toBe(200); // Admin allowed
|
||||
expect(body).toBeInstanceOf(Array);
|
||||
});
|
||||
```
|
||||
|
||||
**Note:** Auth-session requires provider setup in global-setup.ts. See [auth-session configuration](https://seontechnologies.github.io/playwright-utils/auth-session.html).
|
||||
|
||||
**Playwright Utils Benefits for Compliance:**
|
||||
- Multi-user auth testing (regular, admin, etc.)
|
||||
- Token persistence (faster test execution)
|
||||
- Consistent auth patterns (audit trail)
|
||||
- Automatic cleanup
|
||||
|
||||
### Set Higher Quality Thresholds
|
||||
|
||||
**Enterprise quality targets:**
|
||||
- Test coverage: >85% (vs 80% for non-enterprise)
|
||||
- Quality score: >85 (vs 75 for non-enterprise)
|
||||
- P0 coverage: 100% (non-negotiable)
|
||||
- P1 coverage: >95% (vs 90% for non-enterprise)
|
||||
|
||||
**Rationale:** Enterprise systems affect more users, higher stakes.
|
||||
|
||||
### Document Everything
|
||||
|
||||
**Auditors need:**
|
||||
- Why decisions were made (rationale)
|
||||
- Who approved (signatures)
|
||||
- When (timestamps)
|
||||
- What evidence (test results, scan reports)
|
||||
|
||||
**Use TEA's structured outputs:**
|
||||
- Reports have timestamps
|
||||
- Decisions have rationale
|
||||
- Evidence is referenced
|
||||
- Audit trail is automatic
|
||||
|
||||
### Budget for Compliance Testing
|
||||
|
||||
**Enterprise testing costs more:**
|
||||
- Penetration testing: $10k-50k
|
||||
- Security audits: $5k-20k
|
||||
- Performance testing tools: $500-5k/month
|
||||
- Compliance consulting: $200-500/hour
|
||||
|
||||
**Plan accordingly:**
|
||||
- Budget in project cost
|
||||
- Schedule early (3+ months for SOC 2)
|
||||
- Don't skip (non-negotiable for compliance)
|
||||
|
||||
### Use External Validators
|
||||
|
||||
**Don't self-certify:**
|
||||
- Penetration testing: Hire external firm
|
||||
- Security audits: Independent auditor
|
||||
- Compliance: Certification body
|
||||
- Performance: Load testing service
|
||||
|
||||
**TEA's role:** Prepare for external validation, don't replace it.
|
||||
|
||||
## Related Guides
|
||||
|
||||
**Workflow Guides:**
|
||||
- [How to Run NFR Assessment](/docs/how-to/workflows/run-nfr-assess.md) - Deep dive on NFRs
|
||||
- [How to Run Trace](/docs/how-to/workflows/run-trace.md) - Gate decisions with evidence
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md) - Quality audits
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md) - Compliance-focused planning
|
||||
|
||||
**Use-Case Guides:**
|
||||
- [Using TEA with Existing Tests](/docs/how-to/brownfield/use-tea-with-existing-tests.md) - Brownfield patterns
|
||||
|
||||
**Customization:**
|
||||
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) - Production-ready utilities
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [Engagement Models](/docs/explanation/tea/engagement-models.md) - Enterprise model explained
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Probability × impact scoring
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Enterprise quality thresholds
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Complete TEA lifecycle
|
||||
|
||||
## Reference
|
||||
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - All 8 workflows
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Enterprise config options
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Testing patterns
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - TEA terminology
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,577 @@
|
|||
---
|
||||
title: "Using TEA with Existing Tests (Brownfield)"
|
||||
description: Apply TEA workflows to legacy codebases with existing test suites
|
||||
---
|
||||
|
||||
# Using TEA with Existing Tests (Brownfield)
|
||||
|
||||
Use TEA on brownfield projects (existing codebases with legacy tests) to establish coverage baselines, identify gaps, and improve test quality without starting from scratch.
|
||||
|
||||
## When to Use This
|
||||
|
||||
- Existing codebase with some tests already written
|
||||
- Legacy test suite needs quality improvement
|
||||
- Adding features to existing application
|
||||
- Need to understand current test coverage
|
||||
- Want to prevent regression as you add features
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed
|
||||
- TEA agent available
|
||||
- Existing codebase with tests (even if incomplete or low quality)
|
||||
- Tests run successfully (or at least can be executed)
|
||||
|
||||
**Note:** If your codebase is completely undocumented, run `*document-project` first to create baseline documentation.
|
||||
|
||||
## Brownfield Strategy
|
||||
|
||||
### Phase 1: Establish Baseline
|
||||
|
||||
Understand what you have before changing anything.
|
||||
|
||||
#### Step 1: Baseline Coverage with *trace
|
||||
|
||||
Run `*trace` Phase 1 to map existing tests to requirements:
|
||||
|
||||
```
|
||||
*trace
|
||||
```
|
||||
|
||||
**Select:** Phase 1 (Requirements Traceability)
|
||||
|
||||
**Provide:**
|
||||
- Existing requirements docs (PRD, user stories, feature specs)
|
||||
- Test location (`tests/` or wherever tests live)
|
||||
- Focus areas (specific features if large codebase)
|
||||
|
||||
**Output:** `traceability-matrix.md` showing:
|
||||
- Which requirements have tests
|
||||
- Which requirements lack coverage
|
||||
- Coverage classification (FULL/PARTIAL/NONE)
|
||||
- Gap prioritization
|
||||
|
||||
**Example Baseline:**
|
||||
```markdown
|
||||
# Baseline Coverage (Before Improvements)
|
||||
|
||||
**Total Requirements:** 50
|
||||
**Full Coverage:** 15 (30%)
|
||||
**Partial Coverage:** 20 (40%)
|
||||
**No Coverage:** 15 (30%)
|
||||
|
||||
**By Priority:**
|
||||
- P0: 50% coverage (5/10) ❌ Critical gap
|
||||
- P1: 40% coverage (8/20) ⚠️ Needs improvement
|
||||
- P2: 20% coverage (2/10) ✅ Acceptable
|
||||
```
|
||||
|
||||
This baseline becomes your improvement target.
|
||||
|
||||
#### Step 2: Quality Audit with *test-review
|
||||
|
||||
Run `*test-review` on existing tests:
|
||||
|
||||
```
|
||||
*test-review tests/
|
||||
```
|
||||
|
||||
**Output:** `test-review.md` with quality score and issues.
|
||||
|
||||
**Common Brownfield Issues:**
|
||||
- Hard waits everywhere (`page.waitForTimeout(5000)`)
|
||||
- Fragile CSS selectors (`.class > div:nth-child(3)`)
|
||||
- No test isolation (tests depend on execution order)
|
||||
- Try-catch for flow control
|
||||
- Tests don't clean up (leave test data in DB)
|
||||
|
||||
**Example Baseline Quality:**
|
||||
```markdown
|
||||
# Quality Score: 55/100
|
||||
|
||||
**Critical Issues:** 12
|
||||
- 8 hard waits
|
||||
- 4 conditional flow control
|
||||
|
||||
**Recommendations:** 25
|
||||
- Extract fixtures
|
||||
- Improve selectors
|
||||
- Add network assertions
|
||||
```
|
||||
|
||||
This shows where to focus improvement efforts.
|
||||
|
||||
### Phase 2: Prioritize Improvements
|
||||
|
||||
Don't try to fix everything at once.
|
||||
|
||||
#### Focus on Critical Path First
|
||||
|
||||
**Priority 1: P0 Requirements**
|
||||
```
|
||||
Goal: Get P0 coverage to 100%
|
||||
|
||||
Actions:
|
||||
1. Identify P0 requirements with no tests (from trace)
|
||||
2. Run *automate to generate tests for missing P0 scenarios
|
||||
3. Fix critical quality issues in P0 tests (from test-review)
|
||||
```
|
||||
|
||||
**Priority 2: Fix Flaky Tests**
|
||||
```
|
||||
Goal: Eliminate flakiness
|
||||
|
||||
Actions:
|
||||
1. Identify tests with hard waits (from test-review)
|
||||
2. Replace with network-first patterns
|
||||
3. Run burn-in loops to verify stability
|
||||
```
|
||||
|
||||
**Example Modernization:**
|
||||
|
||||
**Before (Flaky - Hard Waits):**
|
||||
```typescript
|
||||
test('checkout completes', async ({ page }) => {
|
||||
await page.click('button[name="checkout"]');
|
||||
await page.waitForTimeout(5000); // ❌ Flaky
|
||||
await expect(page.locator('.confirmation')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**After (Network-First - Vanilla):**
|
||||
```typescript
|
||||
test('checkout completes', async ({ page }) => {
|
||||
const checkoutPromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/checkout') && resp.ok()
|
||||
);
|
||||
await page.click('button[name="checkout"]');
|
||||
await checkoutPromise; // ✅ Deterministic
|
||||
await expect(page.locator('.confirmation')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**After (With Playwright Utils - Cleaner API):**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
|
||||
test('checkout completes', async ({ page, interceptNetworkCall }) => {
|
||||
// Use interceptNetworkCall for cleaner network interception
|
||||
const checkoutCall = interceptNetworkCall({
|
||||
method: 'POST',
|
||||
url: '**/api/checkout'
|
||||
});
|
||||
|
||||
await page.click('button[name="checkout"]');
|
||||
|
||||
// Wait for response (automatic JSON parsing)
|
||||
const { status, responseJson: order } = await checkoutCall;
|
||||
|
||||
// Validate API response
|
||||
expect(status).toBe(200);
|
||||
expect(order.status).toBe('confirmed');
|
||||
|
||||
// Validate UI
|
||||
await expect(page.locator('.confirmation')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Playwright Utils Benefits:**
|
||||
- `interceptNetworkCall` for cleaner network interception
|
||||
- Automatic JSON parsing (`responseJson` ready to use)
|
||||
- No manual `await response.json()`
|
||||
- Glob pattern matching (`**/api/checkout`)
|
||||
- Cleaner, more maintainable code
|
||||
|
||||
**For automatic error detection,** use `network-error-monitor` fixture separately. See [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md#network-error-monitor).
|
||||
|
||||
**Priority 3: P1 Requirements**
|
||||
```
|
||||
Goal: Get P1 coverage to 80%+
|
||||
|
||||
Actions:
|
||||
1. Generate tests for highest-risk P1 gaps
|
||||
2. Improve test quality incrementally
|
||||
```
|
||||
|
||||
#### Create Improvement Roadmap
|
||||
|
||||
```markdown
|
||||
# Test Improvement Roadmap
|
||||
|
||||
## Week 1: Critical Path (P0)
|
||||
- [ ] Add 5 missing P0 tests (Epic 1: Auth)
|
||||
- [ ] Fix 8 hard waits in auth tests
|
||||
- [ ] Verify P0 coverage = 100%
|
||||
|
||||
## Week 2: Flakiness
|
||||
- [ ] Replace all hard waits with network-first
|
||||
- [ ] Fix conditional flow control
|
||||
- [ ] Run burn-in loops (target: 0 failures in 10 runs)
|
||||
|
||||
## Week 3: High-Value Coverage (P1)
|
||||
- [ ] Add 10 missing P1 tests
|
||||
- [ ] Improve selector resilience
|
||||
- [ ] P1 coverage target: 80%
|
||||
|
||||
## Week 4: Quality Polish
|
||||
- [ ] Extract fixtures for common patterns
|
||||
- [ ] Add network assertions
|
||||
- [ ] Quality score target: 75+
|
||||
```
|
||||
|
||||
### Phase 3: Incremental Improvement
|
||||
|
||||
Apply TEA workflows to new work while improving legacy tests.
|
||||
|
||||
#### For New Features (Greenfield Within Brownfield)
|
||||
|
||||
**Use full TEA workflow:**
|
||||
```
|
||||
1. *test-design (epic-level) - Plan tests for new feature
|
||||
2. *atdd - Generate failing tests first (TDD)
|
||||
3. Implement feature
|
||||
4. *automate - Expand coverage
|
||||
5. *test-review - Ensure quality
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- New code has high-quality tests from day one
|
||||
- Gradually raises overall quality
|
||||
- Team learns good patterns
|
||||
|
||||
#### For Bug Fixes (Regression Prevention)
|
||||
|
||||
**Add regression tests:**
|
||||
```
|
||||
1. Reproduce bug with failing test
|
||||
2. Fix bug
|
||||
3. Verify test passes
|
||||
4. Run *test-review on regression test
|
||||
5. Add to regression test suite
|
||||
```
|
||||
|
||||
#### For Refactoring (Regression Safety)
|
||||
|
||||
**Before refactoring:**
|
||||
```
|
||||
1. Run *trace - Baseline coverage
|
||||
2. Note current coverage %
|
||||
3. Refactor code
|
||||
4. Run *trace - Verify coverage maintained
|
||||
5. No coverage should decrease
|
||||
```
|
||||
|
||||
### Phase 4: Continuous Improvement
|
||||
|
||||
Track improvement over time.
|
||||
|
||||
#### Quarterly Quality Audits
|
||||
|
||||
**Q1 Baseline:**
|
||||
```
|
||||
Coverage: 30%
|
||||
Quality Score: 55/100
|
||||
Flakiness: 15% fail rate
|
||||
```
|
||||
|
||||
**Q2 Target:**
|
||||
```
|
||||
Coverage: 50% (focus on P0)
|
||||
Quality Score: 65/100
|
||||
Flakiness: 5%
|
||||
```
|
||||
|
||||
**Q3 Target:**
|
||||
```
|
||||
Coverage: 70%
|
||||
Quality Score: 75/100
|
||||
Flakiness: 1%
|
||||
```
|
||||
|
||||
**Q4 Target:**
|
||||
```
|
||||
Coverage: 85%
|
||||
Quality Score: 85/100
|
||||
Flakiness: <0.5%
|
||||
```
|
||||
|
||||
## Brownfield-Specific Tips
|
||||
|
||||
### Don't Rewrite Everything
|
||||
|
||||
**Common mistake:**
|
||||
```
|
||||
"Our tests are bad, let's delete them all and start over!"
|
||||
```
|
||||
|
||||
**Better approach:**
|
||||
```
|
||||
"Our tests are bad, let's:
|
||||
1. Keep tests that work (even if not perfect)
|
||||
2. Fix critical quality issues incrementally
|
||||
3. Add tests for gaps
|
||||
4. Gradually improve over time"
|
||||
```
|
||||
|
||||
**Why:**
|
||||
- Rewriting is risky (might lose coverage)
|
||||
- Incremental improvement is safer
|
||||
- Team learns gradually
|
||||
- Business value delivered continuously
|
||||
|
||||
### Use Regression Hotspots
|
||||
|
||||
**Identify regression-prone areas:**
|
||||
```markdown
|
||||
## Regression Hotspots
|
||||
|
||||
**Based on:**
|
||||
- Bug reports (last 6 months)
|
||||
- Customer complaints
|
||||
- Code complexity (cyclomatic complexity >10)
|
||||
- Frequent changes (git log analysis)
|
||||
|
||||
**High-Risk Areas:**
|
||||
1. Authentication flow (12 bugs in 6 months)
|
||||
2. Checkout process (8 bugs)
|
||||
3. Payment integration (6 bugs)
|
||||
|
||||
**Test Priority:**
|
||||
- Add regression tests for these areas FIRST
|
||||
- Ensure P0 coverage before touching code
|
||||
```
|
||||
|
||||
### Quarantine Flaky Tests
|
||||
|
||||
Don't let flaky tests block improvement:
|
||||
|
||||
```typescript
|
||||
// Mark flaky tests with .skip temporarily
|
||||
test.skip('flaky test - needs fixing', async ({ page }) => {
|
||||
// TODO: Fix hard wait on line 45
|
||||
// TODO: Add network-first pattern
|
||||
});
|
||||
```
|
||||
|
||||
**Track quarantined tests:**
|
||||
```markdown
|
||||
# Quarantined Tests
|
||||
|
||||
| Test | Reason | Owner | Target Fix Date |
|
||||
| ------------------- | -------------------------- | -------- | --------------- |
|
||||
| checkout.spec.ts:45 | Hard wait causes flakiness | QA Team | 2026-01-20 |
|
||||
| profile.spec.ts:28 | Conditional flow control | Dev Team | 2026-01-25 |
|
||||
```
|
||||
|
||||
**Fix systematically:**
|
||||
- Don't accumulate quarantined tests
|
||||
- Set deadlines for fixes
|
||||
- Review quarantine list weekly
|
||||
|
||||
### Migrate One Directory at a Time
|
||||
|
||||
**Large test suite?** Improve incrementally:
|
||||
|
||||
**Week 1:** `tests/auth/`
|
||||
```
|
||||
1. Run *test-review on auth tests
|
||||
2. Fix critical issues
|
||||
3. Re-review
|
||||
4. Mark directory as "modernized"
|
||||
```
|
||||
|
||||
**Week 2:** `tests/api/`
|
||||
```
|
||||
Same process
|
||||
```
|
||||
|
||||
**Week 3:** `tests/e2e/`
|
||||
```
|
||||
Same process
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Focused improvement
|
||||
- Visible progress
|
||||
- Team learns patterns
|
||||
- Lower risk
|
||||
|
||||
### Document Migration Status
|
||||
|
||||
**Track which tests are modernized:**
|
||||
|
||||
```markdown
|
||||
# Test Suite Status
|
||||
|
||||
| Directory | Tests | Quality Score | Status | Notes |
|
||||
| ------------------ | ----- | ------------- | ------------- | -------------- |
|
||||
| tests/auth/ | 15 | 85/100 | ✅ Modernized | Week 1 cleanup |
|
||||
| tests/api/ | 32 | 78/100 | ⚠️ In Progress | Week 2 |
|
||||
| tests/e2e/ | 28 | 62/100 | ❌ Legacy | Week 3 planned |
|
||||
| tests/integration/ | 12 | 45/100 | ❌ Legacy | Week 4 planned |
|
||||
|
||||
**Legend:**
|
||||
- ✅ Modernized: Quality >80, no critical issues
|
||||
- ⚠️ In Progress: Active improvement
|
||||
- ❌ Legacy: Not yet touched
|
||||
```
|
||||
|
||||
## Common Brownfield Challenges
|
||||
|
||||
### "We Don't Know What Tests Cover"
|
||||
|
||||
**Problem:** No documentation, unclear what tests do.
|
||||
|
||||
**Solution:**
|
||||
```
|
||||
1. Run *trace - TEA analyzes tests and maps to requirements
|
||||
2. Review traceability matrix
|
||||
3. Document findings
|
||||
4. Use as baseline for improvement
|
||||
```
|
||||
|
||||
TEA reverse-engineers test coverage even without documentation.
|
||||
|
||||
### "Tests Are Too Brittle to Touch"
|
||||
|
||||
**Problem:** Afraid to modify tests (might break them).
|
||||
|
||||
**Solution:**
|
||||
```
|
||||
1. Run tests, capture current behavior (baseline)
|
||||
2. Make small improvement (fix one hard wait)
|
||||
3. Run tests again
|
||||
4. If still pass, continue
|
||||
5. If fail, investigate why
|
||||
|
||||
Incremental changes = lower risk
|
||||
```
|
||||
|
||||
### "No One Knows How to Run Tests"
|
||||
|
||||
**Problem:** Test documentation is outdated or missing.
|
||||
|
||||
**Solution:**
|
||||
```
|
||||
1. Document manually or ask TEA to help analyze test structure
|
||||
2. Create tests/README.md with:
|
||||
- How to install dependencies
|
||||
- How to run tests (npx playwright test, npm test, etc.)
|
||||
- What each test directory contains
|
||||
- Common issues and troubleshooting
|
||||
3. Commit documentation for team
|
||||
```
|
||||
|
||||
**Note:** `*framework` is for new test setup, not existing tests. For brownfield, document what you have.
|
||||
|
||||
### "Tests Take Hours to Run"
|
||||
|
||||
**Problem:** Full test suite takes 4+ hours.
|
||||
|
||||
**Solution:**
|
||||
```
|
||||
1. Configure parallel execution (shard tests across workers)
|
||||
2. Add selective testing (run only affected tests on PR)
|
||||
3. Run full suite nightly only
|
||||
4. Optimize slow tests (remove hard waits, improve selectors)
|
||||
|
||||
Before: 4 hours sequential
|
||||
After: 15 minutes with sharding + selective testing
|
||||
```
|
||||
|
||||
**How `*ci` helps:**
|
||||
- Scaffolds CI configuration with parallel sharding examples
|
||||
- Provides selective testing script templates
|
||||
- Documents burn-in and optimization strategies
|
||||
- But YOU configure workers, test selection, and optimization
|
||||
|
||||
**With Playwright Utils burn-in:**
|
||||
- Smart selective testing based on git diff
|
||||
- Volume control (run percentage of affected tests)
|
||||
- See [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md#burn-in)
|
||||
|
||||
### "We Have Tests But They Always Fail"
|
||||
|
||||
**Problem:** Tests are so flaky they're ignored.
|
||||
|
||||
**Solution:**
|
||||
```
|
||||
1. Run *test-review to identify flakiness patterns
|
||||
2. Fix top 5 flaky tests (biggest impact)
|
||||
3. Quarantine remaining flaky tests
|
||||
4. Re-enable as you fix them
|
||||
|
||||
Don't let perfect be the enemy of good
|
||||
```
|
||||
|
||||
## Brownfield TEA Workflow
|
||||
|
||||
### Recommended Sequence
|
||||
|
||||
**1. Documentation (if needed):**
|
||||
```
|
||||
*document-project
|
||||
```
|
||||
|
||||
**2. Baseline (Phase 2):**
|
||||
```
|
||||
*trace Phase 1 - Establish coverage baseline
|
||||
*test-review - Establish quality baseline
|
||||
```
|
||||
|
||||
**3. Planning (Phase 2-3):**
|
||||
```
|
||||
*prd - Document requirements (if missing)
|
||||
*architecture - Document architecture (if missing)
|
||||
*test-design (system-level) - Testability review
|
||||
```
|
||||
|
||||
**4. Infrastructure (Phase 3):**
|
||||
```
|
||||
*framework - Modernize test framework (if needed)
|
||||
*ci - Setup or improve CI/CD
|
||||
```
|
||||
|
||||
**5. Per Epic (Phase 4):**
|
||||
```
|
||||
*test-design (epic-level) - Focus on regression hotspots
|
||||
*automate - Add missing tests
|
||||
*test-review - Ensure quality
|
||||
*trace Phase 1 - Refresh coverage
|
||||
```
|
||||
|
||||
**6. Release Gate:**
|
||||
```
|
||||
*nfr-assess - Validate NFRs (if enterprise)
|
||||
*trace Phase 2 - Gate decision
|
||||
```
|
||||
|
||||
## Related Guides
|
||||
|
||||
**Workflow Guides:**
|
||||
- [How to Run Trace](/docs/how-to/workflows/run-trace.md) - Baseline coverage analysis
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md) - Quality audit
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Fill coverage gaps
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md) - Risk assessment
|
||||
|
||||
**Customization:**
|
||||
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) - Modernize tests with utilities
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [Engagement Models](/docs/explanation/tea/engagement-models.md) - Brownfield model explained
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - What makes tests good
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Fix flakiness
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Prioritize improvements
|
||||
|
||||
## Reference
|
||||
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - All 8 workflows
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Config options
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Testing patterns
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - TEA terminology
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,424 @@
|
|||
---
|
||||
title: "Enable TEA MCP Enhancements"
|
||||
description: Configure Playwright MCP servers for live browser verification during TEA workflows
|
||||
---
|
||||
|
||||
# Enable TEA MCP Enhancements
|
||||
|
||||
Configure Model Context Protocol (MCP) servers to enable live browser verification, exploratory mode, and recording mode in TEA workflows.
|
||||
|
||||
## What are MCP Enhancements?
|
||||
|
||||
MCP (Model Context Protocol) servers enable AI agents to interact with live browsers during test generation. This allows TEA to:
|
||||
|
||||
- **Explore UIs interactively** - Discover actual functionality through browser automation
|
||||
- **Verify selectors** - Generate accurate locators from real DOM
|
||||
- **Validate behavior** - Confirm test scenarios against live applications
|
||||
- **Debug visually** - Use trace viewer and screenshots during generation
|
||||
|
||||
## When to Use This
|
||||
|
||||
**For UI Testing:**
|
||||
- Want exploratory mode in `*test-design` (browser-based UI discovery)
|
||||
- Want recording mode in `*atdd` or `*automate` (verify selectors with live browser)
|
||||
- Want healing mode in `*automate` (fix tests with visual debugging)
|
||||
- Need accurate selectors from actual DOM
|
||||
- Debugging complex UI interactions
|
||||
|
||||
**For API Testing:**
|
||||
- Want healing mode in `*automate` (analyze failures with trace data)
|
||||
- Need to debug test failures (network responses, request/response data, timing)
|
||||
- Want to inspect trace files (network traffic, errors, race conditions)
|
||||
|
||||
**For Both:**
|
||||
- Visual debugging (trace viewer shows network + UI)
|
||||
- Test failure analysis (MCP can run tests and extract errors)
|
||||
- Understanding complex test failures (network + DOM together)
|
||||
|
||||
**Don't use if:**
|
||||
- You don't have MCP servers configured
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed
|
||||
- TEA agent available
|
||||
- IDE with MCP support (Cursor, VS Code with Claude extension)
|
||||
- Node.js v18 or later
|
||||
- Playwright installed
|
||||
|
||||
## Available MCP Servers
|
||||
|
||||
**Two Playwright MCP servers** (actively maintained, continuously updated):
|
||||
|
||||
### 1. Playwright MCP - Browser Automation
|
||||
|
||||
**Command:** `npx @playwright/mcp@latest`
|
||||
|
||||
**Capabilities:**
|
||||
- Navigate to URLs
|
||||
- Click elements
|
||||
- Fill forms
|
||||
- Take screenshots
|
||||
- Extract DOM information
|
||||
|
||||
**Best for:** Exploratory mode, recording mode
|
||||
|
||||
### 2. Playwright Test MCP - Test Runner
|
||||
|
||||
**Command:** `npx playwright run-test-mcp-server`
|
||||
|
||||
**Capabilities:**
|
||||
- Run test files
|
||||
- Analyze failures
|
||||
- Extract error messages
|
||||
- Show trace files
|
||||
|
||||
**Best for:** Healing mode, debugging
|
||||
|
||||
### Recommended: Configure Both
|
||||
|
||||
Both servers work together to provide full TEA MCP capabilities.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Configure MCP Servers
|
||||
|
||||
Add to your IDE's MCP configuration:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"playwright": {
|
||||
"command": "npx",
|
||||
"args": ["@playwright/mcp@latest"]
|
||||
},
|
||||
"playwright-test": {
|
||||
"command": "npx",
|
||||
"args": ["playwright", "run-test-mcp-server"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
See [TEA Overview](/docs/explanation/features/tea-overview.md#playwright-mcp-enhancements) for IDE-specific config locations.
|
||||
|
||||
### 2. Enable in BMAD
|
||||
|
||||
Answer "Yes" when prompted during installation, or set in config:
|
||||
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml
|
||||
tea_use_mcp_enhancements: true
|
||||
```
|
||||
|
||||
### 3. Verify MCPs Running
|
||||
|
||||
Ensure your MCP servers are running in your IDE.
|
||||
|
||||
## How MCP Enhances TEA Workflows
|
||||
|
||||
### *test-design: Exploratory Mode
|
||||
|
||||
**Without MCP:**
|
||||
- TEA infers UI functionality from documentation
|
||||
- Relies on your description of features
|
||||
- May miss actual UI behavior
|
||||
|
||||
**With MCP:**
|
||||
TEA can open live browser to:
|
||||
```
|
||||
"Let me explore the profile page to understand the UI"
|
||||
|
||||
[TEA navigates to /profile]
|
||||
[Takes screenshot]
|
||||
[Extracts accessible elements]
|
||||
|
||||
"I see the profile has:
|
||||
- Name field (editable)
|
||||
- Email field (editable)
|
||||
- Avatar upload button
|
||||
- Save button
|
||||
- Cancel button
|
||||
|
||||
I'll design tests for these interactions."
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Accurate test design based on actual UI
|
||||
- Discovers functionality you might not describe
|
||||
- Validates test scenarios are possible
|
||||
|
||||
### *atdd: Recording Mode
|
||||
|
||||
**Without MCP:**
|
||||
- TEA generates selectors from best practices
|
||||
- TEA infers API patterns from documentation
|
||||
|
||||
**With MCP (Recording Mode):**
|
||||
|
||||
**For UI Tests:**
|
||||
```
|
||||
[TEA navigates to /login with live browser]
|
||||
[Inspects actual form fields]
|
||||
|
||||
"I see:
|
||||
- Email input has label 'Email Address' (not 'Email')
|
||||
- Password input has label 'Your Password'
|
||||
- Submit button has text 'Sign In' (not 'Login')
|
||||
|
||||
I'll use these exact selectors."
|
||||
```
|
||||
|
||||
**For API Tests:**
|
||||
```
|
||||
[TEA analyzes trace files from test runs]
|
||||
[Inspects network requests/responses]
|
||||
|
||||
"I see the API returns:
|
||||
- POST /api/login → 200 with { token, userId }
|
||||
- Response time: 150ms
|
||||
- Required headers: Content-Type, Authorization
|
||||
|
||||
I'll validate these in tests."
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- UI: Accurate selectors from real DOM
|
||||
- API: Validated request/response patterns from trace
|
||||
- Both: Tests work on first run
|
||||
|
||||
### *automate: Healing + Recording Modes
|
||||
|
||||
**Without MCP:**
|
||||
- TEA analyzes test code only
|
||||
- Suggests fixes based on static analysis
|
||||
- Generates tests from documentation/code
|
||||
|
||||
**With MCP:**
|
||||
|
||||
**Healing Mode (UI + API):**
|
||||
```
|
||||
[TEA opens trace file]
|
||||
[Analyzes screenshots + network tab]
|
||||
|
||||
UI failures: "Button selector changed from 'Save' to 'Save Changes'"
|
||||
API failures: "Response structure changed, expected {id} got {userId}"
|
||||
|
||||
[TEA makes fixes]
|
||||
[Verifies with trace analysis]
|
||||
```
|
||||
|
||||
**Recording Mode (UI + API):**
|
||||
```
|
||||
UI: [Inspects actual DOM, generates verified selectors]
|
||||
API: [Analyzes network traffic, validates request/response patterns]
|
||||
|
||||
[Generates tests with verified patterns]
|
||||
[Tests work on first run]
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Visual debugging + trace analysis (not just UI)
|
||||
- Verified selectors (UI) + network patterns (API)
|
||||
- Tests verified against actual application behavior
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Exploratory Mode Example
|
||||
|
||||
```
|
||||
User: Run *test-design
|
||||
|
||||
TEA: What feature are you designing tests for?
|
||||
|
||||
User: The checkout flow at https://shop.example.com
|
||||
|
||||
TEA: [With MCP] Let me explore the checkout flow
|
||||
[Opens browser]
|
||||
[Navigates to checkout]
|
||||
[Takes screenshots]
|
||||
[Identifies all interactive elements]
|
||||
|
||||
TEA: I've explored the checkout flow. I found:
|
||||
- 4-step wizard (Cart → Shipping → Payment → Confirm)
|
||||
- Shipping options (Standard, Express, Overnight)
|
||||
- Payment methods (Credit Card, PayPal)
|
||||
- Promo code field
|
||||
- Order summary with real-time totals
|
||||
|
||||
Test design will cover all these scenarios.
|
||||
```
|
||||
|
||||
### Recording Mode Example
|
||||
|
||||
```
|
||||
User: Run *atdd for user registration feature
|
||||
|
||||
TEA: [With MCP] Let me verify the registration form selectors
|
||||
[Opens browser at /register]
|
||||
[Inspects form]
|
||||
|
||||
TEA: I see the actual form has:
|
||||
- "Full Name" label (not "Name")
|
||||
- "Email Address" label (not "Email")
|
||||
- "Create Password" label (not "Password")
|
||||
- "I agree to Terms" checkbox
|
||||
- "Create Account" button (not "Register")
|
||||
|
||||
Generating tests with verified selectors...
|
||||
```
|
||||
|
||||
### Healing Mode Example
|
||||
|
||||
```
|
||||
User: Run *automate with healing mode
|
||||
|
||||
TEA: [With MCP] Running tests and checking for failures
|
||||
[Runs test suite]
|
||||
[Test fails: selector not found]
|
||||
|
||||
TEA: Test failed on selector '.submit-btn'
|
||||
[Opens trace viewer]
|
||||
[Sees button class changed to '.submit-button']
|
||||
|
||||
Fixing selector and verifying...
|
||||
[Updates test]
|
||||
[Re-runs with MCP]
|
||||
[Test passes]
|
||||
|
||||
Updated test with corrected selector.
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### MCP Servers Not Running
|
||||
|
||||
**Problem:** TEA says MCP enhancements aren't available.
|
||||
|
||||
**Causes:**
|
||||
1. MCP servers not configured in IDE
|
||||
2. Config syntax error in JSON
|
||||
3. IDE not restarted after config
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Verify MCP config file exists
|
||||
ls ~/.cursor/config.json
|
||||
|
||||
# Validate JSON syntax
|
||||
cat ~/.cursor/config.json | python -m json.tool
|
||||
|
||||
# Restart IDE
|
||||
# Cmd+Q (quit) then reopen
|
||||
```
|
||||
|
||||
### Browser Doesn't Open
|
||||
|
||||
**Problem:** MCP enabled but browser never opens.
|
||||
|
||||
**Causes:**
|
||||
1. Playwright browsers not installed
|
||||
2. Headless mode enabled
|
||||
3. MCP server crashed
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Install browsers
|
||||
npx playwright install
|
||||
|
||||
# Check MCP server logs (in IDE)
|
||||
# Look for error messages
|
||||
|
||||
# Try manual MCP server
|
||||
npx @playwright/mcp@latest
|
||||
# Should start without errors
|
||||
```
|
||||
|
||||
### TEA Doesn't Use MCP
|
||||
|
||||
**Problem:** `tea_use_mcp_enhancements: true` but TEA doesn't use browser.
|
||||
|
||||
**Causes:**
|
||||
1. Config not saved
|
||||
2. Workflow run before config update
|
||||
3. MCP servers not running
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Verify config
|
||||
grep tea_use_mcp_enhancements _bmad/bmm/config.yaml
|
||||
# Should show: tea_use_mcp_enhancements: true
|
||||
|
||||
# Restart IDE (reload MCP servers)
|
||||
|
||||
# Start fresh chat (TEA loads config at start)
|
||||
```
|
||||
|
||||
### Selector Verification Fails
|
||||
|
||||
**Problem:** MCP can't find elements TEA is looking for.
|
||||
|
||||
**Causes:**
|
||||
1. Page not fully loaded
|
||||
2. Element behind modal/overlay
|
||||
3. Element requires authentication
|
||||
|
||||
**Solution:**
|
||||
TEA will handle this automatically:
|
||||
- Wait for page load
|
||||
- Dismiss modals if present
|
||||
- Handle auth if needed
|
||||
|
||||
If persistent, provide TEA more context:
|
||||
```
|
||||
"The element is behind a modal - dismiss the modal first"
|
||||
"The page requires login - use credentials X"
|
||||
```
|
||||
|
||||
### MCP Slows Down Workflows
|
||||
|
||||
**Problem:** Workflows take much longer with MCP enabled.
|
||||
|
||||
**Cause:** Browser automation adds overhead.
|
||||
|
||||
**Solution:**
|
||||
Use MCP selectively:
|
||||
- **Enable for:** Complex UIs, new projects, debugging
|
||||
- **Disable for:** Simple features, well-known patterns, API-only testing
|
||||
|
||||
Toggle quickly:
|
||||
```yaml
|
||||
# For this feature (complex UI)
|
||||
tea_use_mcp_enhancements: true
|
||||
|
||||
# For next feature (simple API)
|
||||
tea_use_mcp_enhancements: false
|
||||
```
|
||||
|
||||
## Related Guides
|
||||
|
||||
**Getting Started:**
|
||||
- [TEA Lite Quickstart Tutorial](/docs/tutorials/getting-started/tea-lite-quickstart.md) - Learn TEA basics first
|
||||
|
||||
**Workflow Guides (MCP-Enhanced):**
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md) - Exploratory mode with browser
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Recording mode for accurate selectors
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Healing mode for debugging
|
||||
|
||||
**Other Customization:**
|
||||
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) - Production-ready utilities
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - MCP enhancements in lifecycle
|
||||
- [Engagement Models](/docs/explanation/tea/engagement-models.md) - When to use MCP enhancements
|
||||
|
||||
## Reference
|
||||
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - tea_use_mcp_enhancements option
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - MCP-enhanced workflows
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - MCP Enhancements term
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,813 @@
|
|||
---
|
||||
title: "Integrate Playwright Utils with TEA"
|
||||
description: Add production-ready fixtures and utilities to your TEA-generated tests
|
||||
---
|
||||
|
||||
# Integrate Playwright Utils with TEA
|
||||
|
||||
Integrate `@seontechnologies/playwright-utils` with TEA to get production-ready fixtures, utilities, and patterns in your test suite.
|
||||
|
||||
## What is Playwright Utils?
|
||||
|
||||
A production-ready utility library that provides:
|
||||
- Typed API request helper
|
||||
- Authentication session management
|
||||
- Network recording and replay (HAR)
|
||||
- Network request interception
|
||||
- Async polling (recurse)
|
||||
- Structured logging
|
||||
- File validation (CSV, PDF, XLSX, ZIP)
|
||||
- Burn-in testing utilities
|
||||
- Network error monitoring
|
||||
|
||||
**Repository:** [https://github.com/seontechnologies/playwright-utils](https://github.com/seontechnologies/playwright-utils)
|
||||
|
||||
**npm Package:** `@seontechnologies/playwright-utils`
|
||||
|
||||
## When to Use This
|
||||
|
||||
- You want production-ready fixtures (not DIY)
|
||||
- Your team benefits from standardized patterns
|
||||
- You need utilities like API testing, auth handling, network mocking
|
||||
- You want TEA to generate tests using these utilities
|
||||
- You're building reusable test infrastructure
|
||||
|
||||
**Don't use if:**
|
||||
- You're just learning testing (keep it simple first)
|
||||
- You have your own fixture library
|
||||
- You don't need the utilities
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed
|
||||
- TEA agent available
|
||||
- Test framework setup complete (Playwright)
|
||||
- Node.js v18 or later
|
||||
|
||||
**Note:** Playwright Utils is for Playwright only (not Cypress).
|
||||
|
||||
## Installation
|
||||
|
||||
### Step 1: Install Package
|
||||
|
||||
```bash
|
||||
npm install -D @seontechnologies/playwright-utils
|
||||
```
|
||||
|
||||
### Step 2: Enable in TEA Config
|
||||
|
||||
Edit `_bmad/bmm/config.yaml`:
|
||||
|
||||
```yaml
|
||||
tea_use_playwright_utils: true
|
||||
```
|
||||
|
||||
**Note:** If you enabled this during BMad installation, it's already set.
|
||||
|
||||
### Step 3: Verify Installation
|
||||
|
||||
```bash
|
||||
# Check package installed
|
||||
npm list @seontechnologies/playwright-utils
|
||||
|
||||
# Check TEA config
|
||||
grep tea_use_playwright_utils _bmad/bmm/config.yaml
|
||||
```
|
||||
|
||||
Should show:
|
||||
```
|
||||
@seontechnologies/playwright-utils@2.x.x
|
||||
tea_use_playwright_utils: true
|
||||
```
|
||||
|
||||
## What Changes When Enabled
|
||||
|
||||
### *framework Workflow
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
// Basic Playwright fixtures only
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('api test', async ({ request }) => {
|
||||
const response = await request.get('/api/users');
|
||||
const users = await response.json();
|
||||
expect(response.status()).toBe(200);
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils (Combined Fixtures):**
|
||||
```typescript
|
||||
// All utilities available via single import
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
|
||||
test('api test', async ({ apiRequest, authToken, log }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/users',
|
||||
headers: { Authorization: `Bearer ${authToken}` }
|
||||
});
|
||||
|
||||
log.info('Fetched users', body);
|
||||
expect(status).toBe(200);
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils (Selective Merge):**
|
||||
```typescript
|
||||
import { mergeTests } from '@playwright/test';
|
||||
import { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { test as logFixture } from '@seontechnologies/playwright-utils/log/fixtures';
|
||||
|
||||
export const test = mergeTests(apiRequestFixture, logFixture);
|
||||
export { expect } from '@playwright/test';
|
||||
|
||||
test('api test', async ({ apiRequest, log }) => {
|
||||
log.info('Fetching users');
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/users'
|
||||
});
|
||||
expect(status).toBe(200);
|
||||
});
|
||||
```
|
||||
|
||||
### `*atdd` and `*automate` Workflows
|
||||
|
||||
**Without Playwright Utils:**
|
||||
```typescript
|
||||
// Manual API calls
|
||||
test('should fetch profile', async ({ page, request }) => {
|
||||
const response = await request.get('/api/profile');
|
||||
const profile = await response.json();
|
||||
// Manual parsing and validation
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
|
||||
test('should fetch profile', async ({ apiRequest }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/profile' // 'path' not 'url'
|
||||
}).validateSchema(ProfileSchema); // Chained validation
|
||||
|
||||
expect(status).toBe(200);
|
||||
// body is type-safe: { id: string, name: string, email: string }
|
||||
});
|
||||
```
|
||||
|
||||
### *test-review Workflow
|
||||
|
||||
**Without Playwright Utils:**
|
||||
Reviews against generic Playwright patterns
|
||||
|
||||
**With Playwright Utils:**
|
||||
Reviews against playwright-utils best practices:
|
||||
- Fixture composition patterns
|
||||
- Utility usage (apiRequest, authSession, etc.)
|
||||
- Network-first patterns
|
||||
- Structured logging
|
||||
|
||||
### *ci Workflow
|
||||
|
||||
**Without Playwright Utils:**
|
||||
- Parallel sharding
|
||||
- Burn-in loops (basic shell scripts)
|
||||
- CI triggers (PR, push, schedule)
|
||||
- Artifact collection
|
||||
|
||||
**With Playwright Utils:**
|
||||
Enhanced with smart testing:
|
||||
- Burn-in utility (git diff-based, volume control)
|
||||
- Selective testing (skip config/docs/types changes)
|
||||
- Test prioritization by file changes
|
||||
|
||||
## Available Utilities
|
||||
|
||||
### api-request
|
||||
|
||||
Typed HTTP client with schema validation.
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/api-request.html>
|
||||
|
||||
**Why Use This?**
|
||||
|
||||
| Vanilla Playwright | api-request Utility |
|
||||
|-------------------|---------------------|
|
||||
| Manual `await response.json()` | Automatic JSON parsing |
|
||||
| `response.status()` + separate body parsing | Returns `{ status, body }` structure |
|
||||
| No built-in retry | Automatic retry for 5xx errors |
|
||||
| No schema validation | Single-line `.validateSchema()` |
|
||||
| Verbose status checking | Clean destructuring |
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
import { z } from 'zod';
|
||||
|
||||
const UserSchema = z.object({
|
||||
id: z.string(),
|
||||
name: z.string(),
|
||||
email: z.string().email()
|
||||
});
|
||||
|
||||
test('should create user', async ({ apiRequest }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'POST',
|
||||
path: '/api/users', // Note: 'path' not 'url'
|
||||
body: { name: 'Test User', email: 'test@example.com' } // Note: 'body' not 'data'
|
||||
}).validateSchema(UserSchema); // Chained method (can await separately if needed)
|
||||
|
||||
expect(status).toBe(201);
|
||||
expect(body.id).toBeDefined();
|
||||
expect(body.email).toBe('test@example.com');
|
||||
});
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Returns `{ status, body }` structure
|
||||
- Schema validation with `.validateSchema()` chained method
|
||||
- Automatic retry for 5xx errors
|
||||
- Type-safe response body
|
||||
|
||||
### auth-session
|
||||
|
||||
Authentication session management with token persistence.
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/auth-session.html>
|
||||
|
||||
**Why Use This?**
|
||||
|
||||
| Vanilla Playwright Auth | auth-session |
|
||||
|------------------------|--------------|
|
||||
| Re-authenticate every test run (slow) | Authenticate once, persist to disk |
|
||||
| Single user per setup | Multi-user support (roles, accounts) |
|
||||
| No token expiration handling | Automatic token renewal |
|
||||
| Manual session management | Provider pattern (flexible auth) |
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/auth-session/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
|
||||
test('should access protected route', async ({ page, authToken }) => {
|
||||
// authToken automatically fetched and persisted
|
||||
// No manual login needed - handled by fixture
|
||||
|
||||
await page.goto('/dashboard');
|
||||
await expect(page).toHaveURL('/dashboard');
|
||||
|
||||
// Token is reused across tests (persisted to disk)
|
||||
});
|
||||
```
|
||||
|
||||
**Configuration required** (see auth-session docs for provider setup):
|
||||
```typescript
|
||||
// global-setup.ts
|
||||
import { authStorageInit, setAuthProvider, authGlobalInit } from '@seontechnologies/playwright-utils/auth-session';
|
||||
|
||||
async function globalSetup() {
|
||||
authStorageInit();
|
||||
setAuthProvider(myCustomProvider); // Define your auth mechanism
|
||||
await authGlobalInit(); // Fetch token once
|
||||
}
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Token fetched once, reused across all tests
|
||||
- Persisted to disk (faster subsequent runs)
|
||||
- Multi-user support via `authOptions.userIdentifier`
|
||||
- Automatic token renewal if expired
|
||||
|
||||
### network-recorder
|
||||
|
||||
Record and replay network traffic (HAR) for offline testing.
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/network-recorder.html>
|
||||
|
||||
**Why Use This?**
|
||||
|
||||
| Vanilla Playwright HAR | network-recorder |
|
||||
|------------------------|------------------|
|
||||
| Manual `routeFromHAR()` configuration | Automatic HAR management with `PW_NET_MODE` |
|
||||
| Separate record/playback test files | Same test, switch env var |
|
||||
| No CRUD detection | Stateful mocking (POST/PUT/DELETE work) |
|
||||
| Manual HAR file paths | Auto-organized by test name |
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/network-recorder/fixtures';
|
||||
|
||||
// Record mode: Set environment variable
|
||||
process.env.PW_NET_MODE = 'record';
|
||||
|
||||
test('should work with recorded traffic', async ({ page, context, networkRecorder }) => {
|
||||
// Setup recorder (records or replays based on PW_NET_MODE)
|
||||
await networkRecorder.setup(context);
|
||||
|
||||
// Your normal test code
|
||||
await page.goto('/dashboard');
|
||||
await page.click('#add-item');
|
||||
|
||||
// First run (record): Saves traffic to HAR file
|
||||
// Subsequent runs (playback): Uses HAR file, no backend needed
|
||||
});
|
||||
```
|
||||
|
||||
**Switch modes:**
|
||||
```bash
|
||||
# Record traffic
|
||||
PW_NET_MODE=record npx playwright test
|
||||
|
||||
# Playback traffic (offline)
|
||||
PW_NET_MODE=playback npx playwright test
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Offline testing (no backend needed)
|
||||
- Deterministic responses (same every time)
|
||||
- Faster execution (no network latency)
|
||||
- Stateful mocking (CRUD operations work)
|
||||
|
||||
### intercept-network-call
|
||||
|
||||
Spy or stub network requests with automatic JSON parsing.
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/intercept-network-call.html>
|
||||
|
||||
**Why Use This?**
|
||||
|
||||
| Vanilla Playwright | interceptNetworkCall |
|
||||
|-------------------|----------------------|
|
||||
| Route setup + response waiting (separate steps) | Single declarative call |
|
||||
| Manual `await response.json()` | Automatic JSON parsing (`responseJson`) |
|
||||
| Complex filter predicates | Simple glob patterns (`**/api/**`) |
|
||||
| Verbose syntax | Concise, readable API |
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
|
||||
test('should handle API errors', async ({ page, interceptNetworkCall }) => {
|
||||
// Stub API to return error (set up BEFORE navigation)
|
||||
const profileCall = interceptNetworkCall({
|
||||
method: 'GET',
|
||||
url: '**/api/profile',
|
||||
fulfillResponse: {
|
||||
status: 500,
|
||||
body: { error: 'Server error' }
|
||||
}
|
||||
});
|
||||
|
||||
await page.goto('/profile');
|
||||
|
||||
// Wait for the intercepted response
|
||||
const { status, responseJson } = await profileCall;
|
||||
|
||||
expect(status).toBe(500);
|
||||
expect(responseJson.error).toBe('Server error');
|
||||
await expect(page.getByText('Server error occurred')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Automatic JSON parsing (`responseJson` ready to use)
|
||||
- Spy mode (observe real traffic) or stub mode (mock responses)
|
||||
- Glob pattern URL matching
|
||||
- Returns promise with `{ status, responseJson, requestJson }`
|
||||
|
||||
### recurse
|
||||
|
||||
Async polling for eventual consistency (Cypress-style).
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/recurse.html>
|
||||
|
||||
**Why Use This?**
|
||||
|
||||
| Manual Polling | recurse Utility |
|
||||
|----------------|-----------------|
|
||||
| `while` loops with `waitForTimeout` | Smart polling with exponential backoff |
|
||||
| Hard-coded retry logic | Configurable timeout/interval |
|
||||
| No logging visibility | Optional logging with custom messages |
|
||||
| Verbose, error-prone | Clean, readable API |
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
|
||||
test('should wait for async job completion', async ({ apiRequest, recurse }) => {
|
||||
// Start async job
|
||||
const { body: job } = await apiRequest({
|
||||
method: 'POST',
|
||||
path: '/api/jobs'
|
||||
});
|
||||
|
||||
// Poll until complete (smart waiting)
|
||||
const completed = await recurse(
|
||||
() => apiRequest({ method: 'GET', path: `/api/jobs/${job.id}` }),
|
||||
(result) => result.body.status === 'completed',
|
||||
{
|
||||
timeout: 30000,
|
||||
interval: 2000,
|
||||
log: 'Waiting for job to complete'
|
||||
}
|
||||
});
|
||||
|
||||
expect(completed.body.status).toBe('completed');
|
||||
});
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Smart polling with configurable interval
|
||||
- Handles async jobs, background tasks
|
||||
- Optional logging for debugging
|
||||
- Better than hard waits or manual polling loops
|
||||
|
||||
### log
|
||||
|
||||
Structured logging that integrates with Playwright reports.
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/log.html>
|
||||
|
||||
**Why Use This?**
|
||||
|
||||
| Console.log / print | log Utility |
|
||||
|--------------------|-------------|
|
||||
| Not in test reports | Integrated with Playwright reports |
|
||||
| No step visualization | `.step()` shows in Playwright UI |
|
||||
| Manual object formatting | Logs objects seamlessly |
|
||||
| No structured output | JSON artifacts for debugging |
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { log } from '@seontechnologies/playwright-utils';
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('should login', async ({ page }) => {
|
||||
await log.info('Starting login test');
|
||||
|
||||
await page.goto('/login');
|
||||
await log.step('Navigated to login page'); // Shows in Playwright UI
|
||||
|
||||
await page.getByLabel('Email').fill('test@example.com');
|
||||
await log.debug('Filled email field');
|
||||
|
||||
await log.success('Login completed');
|
||||
// Logs appear in test output and Playwright reports
|
||||
});
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Direct import (no fixture needed for basic usage)
|
||||
- Structured logs in test reports
|
||||
- `.step()` shows in Playwright UI
|
||||
- Logs objects seamlessly (no special handling needed)
|
||||
- Trace test execution
|
||||
|
||||
### file-utils
|
||||
|
||||
Read and validate CSV, PDF, XLSX, ZIP files.
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/file-utils.html>
|
||||
|
||||
**Why Use This?**
|
||||
|
||||
| Vanilla Playwright | file-utils |
|
||||
|-------------------|------------|
|
||||
| ~80 lines per CSV flow | ~10 lines end-to-end |
|
||||
| Manual download event handling | `handleDownload()` encapsulates all |
|
||||
| External parsing libraries | Auto-parsing (CSV, XLSX, PDF, ZIP) |
|
||||
| No validation helpers | Built-in validation (headers, row count) |
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { handleDownload, readCSV } from '@seontechnologies/playwright-utils/file-utils';
|
||||
import { expect } from '@playwright/test';
|
||||
import path from 'node:path';
|
||||
|
||||
const DOWNLOAD_DIR = path.join(__dirname, '../downloads');
|
||||
|
||||
test('should export valid CSV', async ({ page }) => {
|
||||
// Handle download and get file path
|
||||
const downloadPath = await handleDownload({
|
||||
page,
|
||||
downloadDir: DOWNLOAD_DIR,
|
||||
trigger: () => page.click('button:has-text("Export")')
|
||||
});
|
||||
|
||||
// Read and parse CSV
|
||||
const csvResult = await readCSV({ filePath: downloadPath });
|
||||
const { data, headers } = csvResult.content;
|
||||
|
||||
// Validate structure
|
||||
expect(headers).toEqual(['Name', 'Email', 'Status']);
|
||||
expect(data.length).toBeGreaterThan(0);
|
||||
expect(data[0]).toMatchObject({
|
||||
Name: expect.any(String),
|
||||
Email: expect.any(String),
|
||||
Status: expect.any(String)
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Handles downloads automatically
|
||||
- Auto-parses CSV, XLSX, PDF, ZIP
|
||||
- Type-safe access to parsed data
|
||||
- Returns structured `{ headers, data }`
|
||||
|
||||
### burn-in
|
||||
|
||||
Smart test selection with git diff analysis for CI optimization.
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/burn-in.html>
|
||||
|
||||
**Why Use This?**
|
||||
|
||||
| Playwright `--only-changed` | burn-in Utility |
|
||||
|-----------------------------|-----------------|
|
||||
| Config changes trigger all tests | Smart filtering (skip configs, types, docs) |
|
||||
| All or nothing | Volume control (run percentage) |
|
||||
| No customization | Custom dependency analysis |
|
||||
| Slow CI on minor changes | Fast CI with intelligent selection |
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
// scripts/burn-in-changed.ts
|
||||
import { runBurnIn } from '@seontechnologies/playwright-utils/burn-in';
|
||||
|
||||
async function main() {
|
||||
await runBurnIn({
|
||||
configPath: 'playwright.burn-in.config.ts',
|
||||
baseBranch: 'main'
|
||||
});
|
||||
}
|
||||
|
||||
main().catch(console.error);
|
||||
```
|
||||
|
||||
**Config:**
|
||||
```typescript
|
||||
// playwright.burn-in.config.ts
|
||||
import type { BurnInConfig } from '@seontechnologies/playwright-utils/burn-in';
|
||||
|
||||
const config: BurnInConfig = {
|
||||
skipBurnInPatterns: [
|
||||
'**/config/**',
|
||||
'**/*.md',
|
||||
'**/*types*'
|
||||
],
|
||||
burnInTestPercentage: 0.3,
|
||||
burnIn: {
|
||||
repeatEach: 3,
|
||||
retries: 1
|
||||
}
|
||||
};
|
||||
|
||||
export default config;
|
||||
```
|
||||
|
||||
**Package script:**
|
||||
```json
|
||||
{
|
||||
"scripts": {
|
||||
"test:burn-in": "tsx scripts/burn-in-changed.ts"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- **Ensure flake-free tests upfront** - Never deal with test flake again
|
||||
- Smart filtering (skip config, types, docs changes)
|
||||
- Volume control (run percentage of affected tests)
|
||||
- Git diff-based test selection
|
||||
- Faster CI feedback
|
||||
|
||||
### network-error-monitor
|
||||
|
||||
Automatically detect HTTP 4xx/5xx errors during tests.
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/network-error-monitor.html>
|
||||
|
||||
**Why Use This?**
|
||||
|
||||
| Vanilla Playwright | network-error-monitor |
|
||||
|-------------------|----------------------|
|
||||
| UI passes, backend 500 ignored | Auto-fails on any 4xx/5xx |
|
||||
| Manual error checking | Zero boilerplate (auto-enabled) |
|
||||
| Silent failures slip through | Acts like Sentry for tests |
|
||||
| No domino effect prevention | Limits cascading failures |
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
|
||||
|
||||
// That's it! Network monitoring is automatically enabled
|
||||
test('should not have API errors', async ({ page }) => {
|
||||
await page.goto('/dashboard');
|
||||
await page.click('button');
|
||||
|
||||
// Test fails automatically if any HTTP 4xx/5xx errors occur
|
||||
// Error message shows: "Network errors detected: 2 request(s) failed"
|
||||
// GET 500 https://api.example.com/users
|
||||
// POST 503 https://api.example.com/metrics
|
||||
});
|
||||
```
|
||||
|
||||
**Opt-out for validation tests:**
|
||||
```typescript
|
||||
// When testing error scenarios, opt-out with annotation
|
||||
test('should show error message on 404',
|
||||
{ annotation: [{ type: 'skipNetworkMonitoring' }] }, // Array format
|
||||
async ({ page }) => {
|
||||
await page.goto('/invalid-page'); // Will 404
|
||||
await expect(page.getByText('Page not found')).toBeVisible();
|
||||
// Test won't fail on 404 because of annotation
|
||||
}
|
||||
);
|
||||
|
||||
// Or opt-out entire describe block
|
||||
test.describe('error handling',
|
||||
{ annotation: [{ type: 'skipNetworkMonitoring' }] },
|
||||
() => {
|
||||
test('handles 404', async ({ page }) => {
|
||||
// Monitoring disabled for all tests in block
|
||||
});
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Auto-enabled (zero setup)
|
||||
- Catches silent backend failures (500, 503, 504)
|
||||
- **Prevents domino effect** (limits cascading failures from one bad endpoint)
|
||||
- Opt-out with annotations for validation tests
|
||||
- Structured error reporting (JSON artifacts)
|
||||
|
||||
## Fixture Composition
|
||||
|
||||
**Option 1: Use Package's Combined Fixtures (Simplest)**
|
||||
|
||||
```typescript
|
||||
// Import all utilities at once
|
||||
import { test } from '@seontechnologies/playwright-utils/fixtures';
|
||||
import { log } from '@seontechnologies/playwright-utils';
|
||||
import { expect } from '@playwright/test';
|
||||
|
||||
test('api test', async ({ apiRequest, interceptNetworkCall }) => {
|
||||
await log.info('Fetching users');
|
||||
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/users'
|
||||
});
|
||||
|
||||
expect(status).toBe(200);
|
||||
});
|
||||
```
|
||||
|
||||
**Option 2: Create Custom Merged Fixtures (Selective)**
|
||||
|
||||
**File 1: support/merged-fixtures.ts**
|
||||
```typescript
|
||||
import { test as base, mergeTests } from '@playwright/test';
|
||||
import { test as apiRequest } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { test as interceptNetworkCall } from '@seontechnologies/playwright-utils/intercept-network-call/fixtures';
|
||||
import { test as networkErrorMonitor } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
|
||||
import { log } from '@seontechnologies/playwright-utils';
|
||||
|
||||
// Merge only what you need
|
||||
export const test = mergeTests(
|
||||
base,
|
||||
apiRequest,
|
||||
interceptNetworkCall,
|
||||
networkErrorMonitor
|
||||
);
|
||||
|
||||
export const expect = base.expect;
|
||||
export { log };
|
||||
```
|
||||
|
||||
**File 2: tests/api/users.spec.ts**
|
||||
```typescript
|
||||
import { test, expect, log } from '../support/merged-fixtures';
|
||||
|
||||
test('api test', async ({ apiRequest, interceptNetworkCall }) => {
|
||||
await log.info('Fetching users');
|
||||
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/users'
|
||||
});
|
||||
|
||||
expect(status).toBe(200);
|
||||
});
|
||||
```
|
||||
|
||||
**Contrast:**
|
||||
- Option 1: All utilities available, zero setup
|
||||
- Option 2: Pick utilities you need, one central file
|
||||
|
||||
**See working examples:** <https://github.com/seontechnologies/playwright-utils/tree/main/playwright/support>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Import Errors
|
||||
|
||||
**Problem:** Cannot find module '@seontechnologies/playwright-utils/api-request'
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Verify package installed
|
||||
npm list @seontechnologies/playwright-utils
|
||||
|
||||
# Check package.json has correct version
|
||||
"@seontechnologies/playwright-utils": "^2.0.0"
|
||||
|
||||
# Reinstall if needed
|
||||
npm install -D @seontechnologies/playwright-utils
|
||||
```
|
||||
|
||||
### TEA Not Using Utilities
|
||||
|
||||
**Problem:** TEA generates tests without playwright-utils.
|
||||
|
||||
**Causes:**
|
||||
1. Config not set: `tea_use_playwright_utils: false`
|
||||
2. Workflow run before config change
|
||||
3. Package not installed
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check config
|
||||
grep tea_use_playwright_utils _bmad/bmm/config.yaml
|
||||
|
||||
# Should show: tea_use_playwright_utils: true
|
||||
|
||||
# Start fresh chat (TEA loads config at start)
|
||||
```
|
||||
|
||||
### Type Errors with apiRequest
|
||||
|
||||
**Problem:** TypeScript errors on apiRequest response.
|
||||
|
||||
**Cause:** No schema validation.
|
||||
|
||||
**Solution:**
|
||||
```typescript
|
||||
// Add Zod schema for type safety
|
||||
import { z } from 'zod';
|
||||
|
||||
const ProfileSchema = z.object({
|
||||
id: z.string(),
|
||||
name: z.string(),
|
||||
email: z.string().email()
|
||||
});
|
||||
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/profile' // 'path' not 'url'
|
||||
}).validateSchema(ProfileSchema); // Chained method
|
||||
|
||||
expect(status).toBe(200);
|
||||
// body is typed as { id: string, name: string, email: string }
|
||||
```
|
||||
|
||||
## Migration Guide
|
||||
|
||||
## Related Guides
|
||||
|
||||
**Getting Started:**
|
||||
- [TEA Lite Quickstart Tutorial](/docs/tutorials/getting-started/tea-lite-quickstart.md) - Learn TEA basics
|
||||
- [How to Set Up Test Framework](/docs/how-to/workflows/setup-test-framework.md) - Initial framework setup
|
||||
|
||||
**Workflow Guides:**
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Generate tests with utilities
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Expand coverage with utilities
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md) - Review against PW-Utils patterns
|
||||
|
||||
**Other Customization:**
|
||||
- [Enable MCP Enhancements](/docs/how-to/customization/enable-tea-mcp-enhancements.md) - Live browser verification
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - **Why Playwright Utils matters** (part of TEA's three-part solution)
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Pure function → fixture pattern
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Network utilities explained
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Patterns PW-Utils enforces
|
||||
|
||||
## Reference
|
||||
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - tea_use_playwright_utils option
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Playwright Utils fragments
|
||||
- [Glossary](/docs/reference/glossary/index.md#test-architect-tea-concepts) - Playwright Utils term
|
||||
- [Official PW-Utils Docs](https://seontechnologies.github.io/playwright-utils/) - Complete API reference
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -9,7 +9,7 @@ Use the `npx bmad-method install` command to set up BMad in your project with yo
|
|||
|
||||
- Starting a new project with BMad
|
||||
- Adding BMad to an existing codebase
|
||||
- Setting up BMad on a new machine
|
||||
- Update the existing BMad Installation
|
||||
|
||||
:::note[Prerequisites]
|
||||
- **Node.js** 20+ (required for the installer)
|
||||
|
|
@ -29,8 +29,7 @@ npx bmad-method install
|
|||
|
||||
The installer will ask where to install BMad files:
|
||||
|
||||
- Current directory (recommended for new projects)
|
||||
- Subdirectory
|
||||
- Current directory (recommended for new projects if you created the directory yourself and ran from within the directory)
|
||||
- Custom path
|
||||
|
||||
### 3. Select Your AI Tools
|
||||
|
|
@ -40,20 +39,20 @@ Choose which AI tools you'll be using:
|
|||
- Claude Code
|
||||
- Cursor
|
||||
- Windsurf
|
||||
- Other
|
||||
- Many others to choose from
|
||||
|
||||
The installer configures BMad for your selected tools.
|
||||
The installer configures BMad for your selected tools by setting up commands that will call the ui.
|
||||
|
||||
### 4. Choose Modules
|
||||
|
||||
Select which modules to install:
|
||||
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| **BMM** | Core methodology for software development |
|
||||
| **BMGD** | Game development workflows |
|
||||
| **CIS** | Creative intelligence and facilitation |
|
||||
| **BMB** | Building custom agents and workflows |
|
||||
| Module | Purpose |
|
||||
| -------- | ----------------------------------------- |
|
||||
| **BMM** | Core methodology for software development |
|
||||
| **BMGD** | Game development workflows |
|
||||
| **CIS** | Creative intelligence and facilitation |
|
||||
| **BMB** | Building custom agents and workflows |
|
||||
|
||||
### 5. Add Custom Content (Optional)
|
||||
|
||||
|
|
@ -82,11 +81,11 @@ your-project/
|
|||
|
||||
1. Check the `_bmad/` directory exists
|
||||
2. Load an agent in your AI tool
|
||||
3. Run `*menu` to see available commands
|
||||
3. Run `/workflow-init` which will autocomplete to the full command to see available commands
|
||||
|
||||
## Configuration
|
||||
|
||||
Edit `_bmad/[module]/config.yaml` to customize:
|
||||
Edit `_bmad/[module]/config.yaml` to customize. For example these could be changed:
|
||||
|
||||
```yaml
|
||||
output_folder: ./_bmad-output
|
||||
|
|
|
|||
|
|
@ -215,6 +215,5 @@ When reporting issues, include:
|
|||
|
||||
## Next Steps
|
||||
|
||||
- **[Quick Start Guide](/docs/tutorials/getting-started/quick-start-bmgd.md)** - Getting started
|
||||
- **[Workflows Guide](/docs/reference/workflows/index.md)** - Workflow reference
|
||||
- **[Glossary](/docs/reference/glossary/index.md)** - Terminology
|
||||
|
|
|
|||
|
|
@ -0,0 +1,436 @@
|
|||
---
|
||||
title: "How to Run ATDD with TEA"
|
||||
description: Generate failing acceptance tests before implementation using TEA's ATDD workflow
|
||||
---
|
||||
|
||||
# How to Run ATDD with TEA
|
||||
|
||||
Use TEA's `*atdd` workflow to generate failing acceptance tests BEFORE implementation. This is the TDD (Test-Driven Development) red phase - tests fail first, guide development, then pass.
|
||||
|
||||
## When to Use This
|
||||
|
||||
- You're about to implement a NEW feature (feature doesn't exist yet)
|
||||
- You want to follow TDD workflow (red → green → refactor)
|
||||
- You want tests to guide your implementation
|
||||
- You're practicing acceptance test-driven development
|
||||
|
||||
**Don't use this if:**
|
||||
- Feature already exists (use `*automate` instead)
|
||||
- You want tests that pass immediately
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed
|
||||
- TEA agent available
|
||||
- Test framework setup complete (run `*framework` if needed)
|
||||
- Story or feature defined with acceptance criteria
|
||||
|
||||
**Note:** This guide uses Playwright examples. If using Cypress, commands and syntax will differ (e.g., `cy.get()` instead of `page.locator()`).
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Load TEA Agent
|
||||
|
||||
Start a fresh chat and load TEA:
|
||||
|
||||
```
|
||||
*tea
|
||||
```
|
||||
|
||||
### 2. Run the ATDD Workflow
|
||||
|
||||
```
|
||||
*atdd
|
||||
```
|
||||
|
||||
### 3. Provide Context
|
||||
|
||||
TEA will ask for:
|
||||
|
||||
**Story/Feature Details:**
|
||||
```
|
||||
We're adding a user profile page where users can:
|
||||
- View their profile information
|
||||
- Edit their name and email
|
||||
- Upload a profile picture
|
||||
- Save changes with validation
|
||||
```
|
||||
|
||||
**Acceptance Criteria:**
|
||||
```
|
||||
Given I'm logged in
|
||||
When I navigate to /profile
|
||||
Then I see my current name and email
|
||||
|
||||
Given I'm on the profile page
|
||||
When I click "Edit Profile"
|
||||
Then I can modify my name and email
|
||||
|
||||
Given I've edited my profile
|
||||
When I click "Save"
|
||||
Then my changes are persisted
|
||||
And I see a success message
|
||||
|
||||
Given I upload an invalid file type
|
||||
When I try to save
|
||||
Then I see an error message
|
||||
And changes are not saved
|
||||
```
|
||||
|
||||
**Reference Documents** (optional):
|
||||
- Point to your story file
|
||||
- Reference PRD or tech spec
|
||||
- Link to test design (if you ran `*test-design` first)
|
||||
|
||||
### 4. Specify Test Levels
|
||||
|
||||
TEA will ask what test levels to generate:
|
||||
|
||||
**Options:**
|
||||
- E2E tests (browser-based, full user journey)
|
||||
- API tests (backend only, faster)
|
||||
- Component tests (UI components in isolation)
|
||||
- Mix of levels (see [API Tests First, E2E Later](#api-tests-first-e2e-later) tip)
|
||||
|
||||
### Component Testing by Framework
|
||||
|
||||
TEA generates component tests using framework-appropriate tools:
|
||||
|
||||
| Your Framework | Component Testing Tool |
|
||||
| -------------- | ------------------------------------------- |
|
||||
| **Cypress** | Cypress Component Testing (*.cy.tsx) |
|
||||
| **Playwright** | Vitest + React Testing Library (*.test.tsx) |
|
||||
|
||||
**Example response:**
|
||||
```
|
||||
Generate:
|
||||
- API tests for profile CRUD operations
|
||||
- E2E tests for the complete profile editing flow
|
||||
- Component tests for ProfileForm validation (if using Cypress or Vitest)
|
||||
- Focus on P0 and P1 scenarios
|
||||
```
|
||||
|
||||
### 5. Review Generated Tests
|
||||
|
||||
TEA generates **failing tests** in appropriate directories:
|
||||
|
||||
#### API Tests (`tests/api/profile.spec.ts`):
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test.describe('Profile API', () => {
|
||||
test('should fetch user profile', async ({ request }) => {
|
||||
const response = await request.get('/api/profile');
|
||||
|
||||
expect(response.status()).toBe(200);
|
||||
const profile = await response.json();
|
||||
expect(profile).toHaveProperty('name');
|
||||
expect(profile).toHaveProperty('email');
|
||||
expect(profile).toHaveProperty('avatarUrl');
|
||||
});
|
||||
|
||||
test('should update user profile', async ({ request }) => {
|
||||
const response = await request.patch('/api/profile', {
|
||||
data: {
|
||||
name: 'Updated Name',
|
||||
email: 'updated@example.com'
|
||||
}
|
||||
});
|
||||
|
||||
expect(response.status()).toBe(200);
|
||||
const updated = await response.json();
|
||||
expect(updated.name).toBe('Updated Name');
|
||||
expect(updated.email).toBe('updated@example.com');
|
||||
});
|
||||
|
||||
test('should validate email format', async ({ request }) => {
|
||||
const response = await request.patch('/api/profile', {
|
||||
data: {
|
||||
email: 'invalid-email'
|
||||
}
|
||||
});
|
||||
|
||||
expect(response.status()).toBe(400);
|
||||
const error = await response.json();
|
||||
expect(error.message).toContain('Invalid email format');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
import { z } from 'zod';
|
||||
|
||||
const ProfileSchema = z.object({
|
||||
name: z.string(),
|
||||
email: z.string().email(),
|
||||
avatarUrl: z.string().url()
|
||||
});
|
||||
|
||||
test.describe('Profile API', () => {
|
||||
test('should fetch user profile', async ({ apiRequest }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/profile'
|
||||
}).validateSchema(ProfileSchema); // Chained validation
|
||||
|
||||
expect(status).toBe(200);
|
||||
// Schema already validated, type-safe access
|
||||
expect(body.name).toBeDefined();
|
||||
expect(body.email).toContain('@');
|
||||
});
|
||||
|
||||
test('should update user profile', async ({ apiRequest }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'PATCH',
|
||||
path: '/api/profile',
|
||||
body: {
|
||||
name: 'Updated Name',
|
||||
email: 'updated@example.com'
|
||||
}
|
||||
}).validateSchema(ProfileSchema); // Chained validation
|
||||
|
||||
expect(status).toBe(200);
|
||||
expect(body.name).toBe('Updated Name');
|
||||
expect(body.email).toBe('updated@example.com');
|
||||
});
|
||||
|
||||
test('should validate email format', async ({ apiRequest }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'PATCH',
|
||||
path: '/api/profile',
|
||||
body: { email: 'invalid-email' }
|
||||
});
|
||||
|
||||
expect(status).toBe(400);
|
||||
expect(body.message).toContain('Invalid email format');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Key Benefits:**
|
||||
- Returns `{ status, body }` (cleaner than `response.status()` + `await response.json()`)
|
||||
- Automatic schema validation with Zod
|
||||
- Type-safe response bodies
|
||||
- Automatic retry for 5xx errors
|
||||
- Less boilerplate
|
||||
|
||||
#### E2E Tests (`tests/e2e/profile.spec.ts`):
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('should edit and save profile', async ({ page }) => {
|
||||
// Login first
|
||||
await page.goto('/login');
|
||||
await page.getByLabel('Email').fill('test@example.com');
|
||||
await page.getByLabel('Password').fill('password123');
|
||||
await page.getByRole('button', { name: 'Sign in' }).click();
|
||||
|
||||
// Navigate to profile
|
||||
await page.goto('/profile');
|
||||
|
||||
// Edit profile
|
||||
await page.getByRole('button', { name: 'Edit Profile' }).click();
|
||||
await page.getByLabel('Name').fill('Updated Name');
|
||||
await page.getByRole('button', { name: 'Save' }).click();
|
||||
|
||||
// Verify success
|
||||
await expect(page.getByText('Profile updated')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
TEA generates additional E2E tests for display, validation errors, etc. based on acceptance criteria.
|
||||
|
||||
#### Implementation Checklist
|
||||
|
||||
TEA also provides an implementation checklist:
|
||||
|
||||
```markdown
|
||||
## Implementation Checklist
|
||||
|
||||
### Backend
|
||||
- [ ] Create `GET /api/profile` endpoint
|
||||
- [ ] Create `PATCH /api/profile` endpoint
|
||||
- [ ] Add email validation middleware
|
||||
- [ ] Add profile picture upload handling
|
||||
- [ ] Write API unit tests
|
||||
|
||||
### Frontend
|
||||
- [ ] Create ProfilePage component
|
||||
- [ ] Implement profile form with validation
|
||||
- [ ] Add file upload for avatar
|
||||
- [ ] Handle API errors gracefully
|
||||
- [ ] Add loading states
|
||||
|
||||
### Tests
|
||||
- [x] API tests generated (failing)
|
||||
- [x] E2E tests generated (failing)
|
||||
- [ ] Run tests after implementation (should pass)
|
||||
```
|
||||
|
||||
### 6. Verify Tests Fail
|
||||
|
||||
This is the TDD red phase - tests MUST fail before implementation.
|
||||
|
||||
**For Playwright:**
|
||||
```bash
|
||||
npx playwright test
|
||||
```
|
||||
|
||||
**For Cypress:**
|
||||
```bash
|
||||
npx cypress run
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
Running 6 tests using 1 worker
|
||||
|
||||
✗ tests/api/profile.spec.ts:3:3 › should fetch user profile
|
||||
Error: expect(received).toBe(expected)
|
||||
Expected: 200
|
||||
Received: 404
|
||||
|
||||
✗ tests/e2e/profile.spec.ts:10:3 › should display current profile information
|
||||
Error: page.goto: net::ERR_ABORTED
|
||||
```
|
||||
|
||||
**All tests should fail!** This confirms:
|
||||
- Feature doesn't exist yet
|
||||
- Tests will guide implementation
|
||||
- You have clear success criteria
|
||||
|
||||
### 7. Implement the Feature
|
||||
|
||||
Now implement the feature following the test guidance:
|
||||
|
||||
1. Start with API tests (backend first)
|
||||
2. Make API tests pass
|
||||
3. Move to E2E tests (frontend)
|
||||
4. Make E2E tests pass
|
||||
5. Refactor with confidence (tests protect you)
|
||||
|
||||
### 8. Verify Tests Pass
|
||||
|
||||
After implementation, run your test suite.
|
||||
|
||||
**For Playwright:**
|
||||
```bash
|
||||
npx playwright test
|
||||
```
|
||||
|
||||
**For Cypress:**
|
||||
```bash
|
||||
npx cypress run
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
Running 6 tests using 1 worker
|
||||
|
||||
✓ tests/api/profile.spec.ts:3:3 › should fetch user profile (850ms)
|
||||
✓ tests/api/profile.spec.ts:15:3 › should update user profile (1.2s)
|
||||
✓ tests/api/profile.spec.ts:30:3 › should validate email format (650ms)
|
||||
✓ tests/e2e/profile.spec.ts:10:3 › should display current profile (2.1s)
|
||||
✓ tests/e2e/profile.spec.ts:18:3 › should edit and save profile (3.2s)
|
||||
✓ tests/e2e/profile.spec.ts:35:3 › should show validation error (1.8s)
|
||||
|
||||
6 passed (9.8s)
|
||||
```
|
||||
|
||||
**Green!** You've completed the TDD cycle: red → green → refactor.
|
||||
|
||||
## What You Get
|
||||
|
||||
### Failing Tests
|
||||
- API tests for backend endpoints
|
||||
- E2E tests for user workflows
|
||||
- Component tests (if requested)
|
||||
- All tests fail initially (red phase)
|
||||
|
||||
### Implementation Guidance
|
||||
- Clear checklist of what to build
|
||||
- Acceptance criteria translated to assertions
|
||||
- Edge cases and error scenarios identified
|
||||
|
||||
### TDD Workflow Support
|
||||
- Tests guide implementation
|
||||
- Confidence to refactor
|
||||
- Living documentation of features
|
||||
|
||||
## Tips
|
||||
|
||||
### Start with Test Design
|
||||
|
||||
Run `*test-design` before `*atdd` for better results:
|
||||
|
||||
```
|
||||
*test-design # Risk assessment and priorities
|
||||
*atdd # Generate tests based on design
|
||||
```
|
||||
|
||||
### MCP Enhancements (Optional)
|
||||
|
||||
If you have MCP servers configured (`tea_use_mcp_enhancements: true`), TEA can use them during `*atdd`.
|
||||
|
||||
**Note:** ATDD is for features that don't exist yet, so recording mode (verify selectors with live UI) only applies if you have skeleton/mockup UI already implemented. For typical ATDD (no UI yet), TEA infers selectors from best practices.
|
||||
|
||||
See [Enable MCP Enhancements](/docs/how-to/customization/enable-tea-mcp-enhancements.md) for setup.
|
||||
|
||||
### Focus on P0/P1 Scenarios
|
||||
|
||||
Don't generate tests for everything at once:
|
||||
|
||||
```
|
||||
Generate tests for:
|
||||
- P0: Critical path (happy path)
|
||||
- P1: High value (validation, errors)
|
||||
|
||||
Skip P2/P3 for now - add later with *automate
|
||||
```
|
||||
|
||||
### API Tests First, E2E Later
|
||||
|
||||
Recommended order:
|
||||
1. Generate API tests with `*atdd`
|
||||
2. Implement backend (make API tests pass)
|
||||
3. Generate E2E tests with `*atdd` (or `*automate`)
|
||||
4. Implement frontend (make E2E tests pass)
|
||||
|
||||
This "outside-in" approach is faster and more reliable.
|
||||
|
||||
### Keep Tests Deterministic
|
||||
|
||||
TEA generates deterministic tests by default:
|
||||
- No hard waits (`waitForTimeout`)
|
||||
- Network-first patterns (wait for responses)
|
||||
- Explicit assertions (no conditionals)
|
||||
|
||||
Don't modify these patterns - they prevent flakiness!
|
||||
|
||||
## Related Guides
|
||||
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md) - Plan before generating
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Tests for existing features
|
||||
- [How to Set Up Test Framework](/docs/how-to/workflows/setup-test-framework.md) - Initial setup
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - **Why TEA generates quality tests** (foundational)
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Why P0 vs P3 matters
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - What makes tests good
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Avoiding flakiness
|
||||
|
||||
## Reference
|
||||
|
||||
- [Command: *atdd](/docs/reference/tea/commands.md#atdd) - Full command reference
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - MCP and Playwright Utils options
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,653 @@
|
|||
---
|
||||
title: "How to Run Automate with TEA"
|
||||
description: Expand test automation coverage after implementation using TEA's automate workflow
|
||||
---
|
||||
|
||||
# How to Run Automate with TEA
|
||||
|
||||
Use TEA's `*automate` workflow to generate comprehensive tests for existing features. Unlike `*atdd`, these tests pass immediately because the feature already exists.
|
||||
|
||||
## When to Use This
|
||||
|
||||
- Feature already exists and works
|
||||
- Want to add test coverage to existing code
|
||||
- Need tests that pass immediately
|
||||
- Expanding existing test suite
|
||||
- Adding tests to legacy code
|
||||
|
||||
**Don't use this if:**
|
||||
- Feature doesn't exist yet (use `*atdd` instead)
|
||||
- Want failing tests to guide development (use `*atdd` for TDD)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed
|
||||
- TEA agent available
|
||||
- Test framework setup complete (run `*framework` if needed)
|
||||
- Feature implemented and working
|
||||
|
||||
**Note:** This guide uses Playwright examples. If using Cypress, commands and syntax will differ.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Load TEA Agent
|
||||
|
||||
Start a fresh chat and load TEA:
|
||||
|
||||
```
|
||||
*tea
|
||||
```
|
||||
|
||||
### 2. Run the Automate Workflow
|
||||
|
||||
```
|
||||
*automate
|
||||
```
|
||||
|
||||
### 3. Provide Context
|
||||
|
||||
TEA will ask for context about what you're testing.
|
||||
|
||||
#### Option A: BMad-Integrated Mode (Recommended)
|
||||
|
||||
If you have BMad artifacts (stories, test designs, PRDs):
|
||||
|
||||
**What are you testing?**
|
||||
```
|
||||
I'm testing the user profile feature we just implemented.
|
||||
Story: story-profile-management.md
|
||||
Test Design: test-design-epic-1.md
|
||||
```
|
||||
|
||||
**Reference documents:**
|
||||
- Story file with acceptance criteria
|
||||
- Test design document (if available)
|
||||
- PRD sections relevant to this feature
|
||||
- Tech spec (if available)
|
||||
|
||||
**Existing tests:**
|
||||
```
|
||||
We have basic tests in tests/e2e/profile-view.spec.ts
|
||||
Avoid duplicating that coverage
|
||||
```
|
||||
|
||||
TEA will analyze your artifacts and generate comprehensive tests that:
|
||||
- Cover acceptance criteria from the story
|
||||
- Follow priorities from test design (P0 → P1 → P2)
|
||||
- Avoid duplicating existing tests
|
||||
- Include edge cases and error scenarios
|
||||
|
||||
#### Option B: Standalone Mode
|
||||
|
||||
If you're using TEA Solo or don't have BMad artifacts:
|
||||
|
||||
**What are you testing?**
|
||||
```
|
||||
TodoMVC React application at https://todomvc.com/examples/react/dist/
|
||||
Features: Create todos, mark as complete, filter by status, delete todos
|
||||
```
|
||||
|
||||
**Specific scenarios to cover:**
|
||||
```
|
||||
- Creating todos (happy path)
|
||||
- Marking todos as complete/incomplete
|
||||
- Filtering (All, Active, Completed)
|
||||
- Deleting todos
|
||||
- Edge cases (empty input, long text)
|
||||
```
|
||||
|
||||
TEA will analyze the application and generate tests based on your description.
|
||||
|
||||
### 4. Specify Test Levels
|
||||
|
||||
TEA will ask which test levels to generate:
|
||||
|
||||
**Options:**
|
||||
- **E2E tests** - Full browser-based user workflows
|
||||
- **API tests** - Backend endpoint testing (faster, more reliable)
|
||||
- **Component tests** - UI component testing in isolation (framework-dependent)
|
||||
- **Mix** - Combination of levels (recommended)
|
||||
|
||||
**Example response:**
|
||||
```
|
||||
Generate:
|
||||
- API tests for all CRUD operations
|
||||
- E2E tests for critical user workflows (P0)
|
||||
- Focus on P0 and P1 scenarios
|
||||
- Skip P3 (low priority edge cases)
|
||||
```
|
||||
|
||||
### 5. Review Generated Tests
|
||||
|
||||
TEA generates a comprehensive test suite with multiple test levels.
|
||||
|
||||
#### API Tests (`tests/api/profile.spec.ts`):
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test.describe('Profile API', () => {
|
||||
let authToken: string;
|
||||
|
||||
test.beforeAll(async ({ request }) => {
|
||||
// Manual auth token fetch
|
||||
const response = await request.post('/api/auth/login', {
|
||||
data: { email: 'test@example.com', password: 'password123' }
|
||||
});
|
||||
const { token } = await response.json();
|
||||
authToken = token;
|
||||
});
|
||||
|
||||
test('should fetch user profile', async ({ request }) => {
|
||||
const response = await request.get('/api/profile', {
|
||||
headers: { Authorization: `Bearer ${authToken}` }
|
||||
});
|
||||
|
||||
expect(response.ok()).toBeTruthy();
|
||||
const profile = await response.json();
|
||||
expect(profile).toMatchObject({
|
||||
id: expect.any(String),
|
||||
name: expect.any(String),
|
||||
email: expect.any(String)
|
||||
});
|
||||
});
|
||||
|
||||
test('should update profile successfully', async ({ request }) => {
|
||||
const response = await request.patch('/api/profile', {
|
||||
headers: { Authorization: `Bearer ${authToken}` },
|
||||
data: {
|
||||
name: 'Updated Name',
|
||||
bio: 'Test bio'
|
||||
}
|
||||
});
|
||||
|
||||
expect(response.ok()).toBeTruthy();
|
||||
const updated = await response.json();
|
||||
expect(updated.name).toBe('Updated Name');
|
||||
expect(updated.bio).toBe('Test bio');
|
||||
});
|
||||
|
||||
test('should validate email format', async ({ request }) => {
|
||||
const response = await request.patch('/api/profile', {
|
||||
headers: { Authorization: `Bearer ${authToken}` },
|
||||
data: { email: 'invalid-email' }
|
||||
});
|
||||
|
||||
expect(response.status()).toBe(400);
|
||||
const error = await response.json();
|
||||
expect(error.message).toContain('Invalid email');
|
||||
});
|
||||
|
||||
test('should require authentication', async ({ request }) => {
|
||||
const response = await request.get('/api/profile');
|
||||
expect(response.status()).toBe(401);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
import { test as base, expect } from '@playwright/test';
|
||||
import { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { createAuthFixtures } from '@seontechnologies/playwright-utils/auth-session';
|
||||
import { mergeTests } from '@playwright/test';
|
||||
import { z } from 'zod';
|
||||
|
||||
const ProfileSchema = z.object({
|
||||
id: z.string(),
|
||||
name: z.string(),
|
||||
email: z.string().email()
|
||||
});
|
||||
|
||||
// Merge API and auth fixtures
|
||||
const authFixtureTest = base.extend(createAuthFixtures());
|
||||
export const testWithAuth = mergeTests(apiRequestFixture, authFixtureTest);
|
||||
|
||||
testWithAuth.describe('Profile API', () => {
|
||||
testWithAuth('should fetch user profile', async ({ apiRequest, authToken }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'GET',
|
||||
path: '/api/profile',
|
||||
headers: { Authorization: `Bearer ${authToken}` }
|
||||
}).validateSchema(ProfileSchema); // Chained validation
|
||||
|
||||
expect(status).toBe(200);
|
||||
// Schema already validated, type-safe access
|
||||
expect(body.name).toBeDefined();
|
||||
});
|
||||
|
||||
testWithAuth('should update profile successfully', async ({ apiRequest, authToken }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'PATCH',
|
||||
path: '/api/profile',
|
||||
body: { name: 'Updated Name', bio: 'Test bio' },
|
||||
headers: { Authorization: `Bearer ${authToken}` }
|
||||
}).validateSchema(ProfileSchema); // Chained validation
|
||||
|
||||
expect(status).toBe(200);
|
||||
expect(body.name).toBe('Updated Name');
|
||||
});
|
||||
|
||||
testWithAuth('should validate email format', async ({ apiRequest, authToken }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'PATCH',
|
||||
path: '/api/profile',
|
||||
body: { email: 'invalid-email' },
|
||||
headers: { Authorization: `Bearer ${authToken}` }
|
||||
});
|
||||
|
||||
expect(status).toBe(400);
|
||||
expect(body.message).toContain('Invalid email');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Key Differences:**
|
||||
- `authToken` fixture (persisted, reused across tests)
|
||||
- `apiRequest` returns `{ status, body }` (cleaner)
|
||||
- Schema validation with Zod (type-safe)
|
||||
- Automatic retry for 5xx errors
|
||||
- Less boilerplate (no manual `await response.json()` everywhere)
|
||||
|
||||
#### E2E Tests (`tests/e2e/profile.spec.ts`):
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('should edit profile', async ({ page }) => {
|
||||
// Login
|
||||
await page.goto('/login');
|
||||
await page.getByLabel('Email').fill('test@example.com');
|
||||
await page.getByLabel('Password').fill('password123');
|
||||
await page.getByRole('button', { name: 'Sign in' }).click();
|
||||
|
||||
// Edit profile
|
||||
await page.goto('/profile');
|
||||
await page.getByRole('button', { name: 'Edit Profile' }).click();
|
||||
await page.getByLabel('Name').fill('New Name');
|
||||
await page.getByRole('button', { name: 'Save' }).click();
|
||||
|
||||
// Verify success
|
||||
await expect(page.getByText('Profile updated')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
TEA generates additional tests for validation, edge cases, etc. based on priorities.
|
||||
|
||||
#### Fixtures (`tests/support/fixtures/profile.ts`):
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
import { test as base, Page } from '@playwright/test';
|
||||
|
||||
type ProfileFixtures = {
|
||||
authenticatedPage: Page;
|
||||
testProfile: {
|
||||
name: string;
|
||||
email: string;
|
||||
bio: string;
|
||||
};
|
||||
};
|
||||
|
||||
export const test = base.extend<ProfileFixtures>({
|
||||
authenticatedPage: async ({ page }, use) => {
|
||||
// Manual login flow
|
||||
await page.goto('/login');
|
||||
await page.getByLabel('Email').fill('test@example.com');
|
||||
await page.getByLabel('Password').fill('password123');
|
||||
await page.getByRole('button', { name: 'Sign in' }).click();
|
||||
await page.waitForURL(/\/dashboard/);
|
||||
|
||||
await use(page);
|
||||
},
|
||||
|
||||
testProfile: async ({ request }, use) => {
|
||||
// Static test data
|
||||
const profile = {
|
||||
name: 'Test User',
|
||||
email: 'test@example.com',
|
||||
bio: 'Test bio'
|
||||
};
|
||||
|
||||
await use(profile);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
import { test as base } from '@playwright/test';
|
||||
import { createAuthFixtures } from '@seontechnologies/playwright-utils/auth-session';
|
||||
import { mergeTests } from '@playwright/test';
|
||||
import { faker } from '@faker-js/faker';
|
||||
|
||||
type ProfileFixtures = {
|
||||
testProfile: {
|
||||
name: string;
|
||||
email: string;
|
||||
bio: string;
|
||||
};
|
||||
};
|
||||
|
||||
// Merge auth fixtures with custom fixtures
|
||||
const authTest = base.extend(createAuthFixtures());
|
||||
const profileTest = base.extend<ProfileFixtures>({
|
||||
testProfile: async ({}, use) => {
|
||||
// Dynamic test data with faker
|
||||
const profile = {
|
||||
name: faker.person.fullName(),
|
||||
email: faker.internet.email(),
|
||||
bio: faker.person.bio()
|
||||
};
|
||||
|
||||
await use(profile);
|
||||
}
|
||||
});
|
||||
|
||||
export const test = mergeTests(authTest, profileTest);
|
||||
export { expect } from '@playwright/test';
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```typescript
|
||||
import { test, expect } from '../support/fixtures/profile';
|
||||
|
||||
test('should update profile', async ({ page, authToken, testProfile }) => {
|
||||
// authToken from auth-session (automatic, persisted)
|
||||
// testProfile from custom fixture (dynamic data)
|
||||
|
||||
await page.goto('/profile');
|
||||
// Test with dynamic, unique data
|
||||
});
|
||||
```
|
||||
|
||||
**Key Benefits:**
|
||||
- `authToken` fixture (persisted token, no manual login)
|
||||
- Dynamic test data with faker (no conflicts)
|
||||
- Fixture composition with mergeTests
|
||||
- Reusable across test files
|
||||
|
||||
### 6. Review Additional Artifacts
|
||||
|
||||
TEA also generates:
|
||||
|
||||
#### Updated README (`tests/README.md`):
|
||||
|
||||
```markdown
|
||||
# Test Suite
|
||||
|
||||
## Running Tests
|
||||
|
||||
### All Tests
|
||||
npm test
|
||||
|
||||
### Specific Levels
|
||||
npm run test:api # API tests only
|
||||
npm run test:e2e # E2E tests only
|
||||
npm run test:smoke # Smoke tests (@smoke tag)
|
||||
|
||||
### Single File
|
||||
npx playwright test tests/api/profile.spec.ts
|
||||
|
||||
## Test Structure
|
||||
|
||||
tests/
|
||||
├── api/ # API tests (fast, reliable)
|
||||
├── e2e/ # E2E tests (full workflows)
|
||||
├── fixtures/ # Shared test utilities
|
||||
└── README.md
|
||||
|
||||
## Writing Tests
|
||||
|
||||
Follow the patterns in existing tests:
|
||||
- Use fixtures for authentication
|
||||
- Network-first patterns (no hard waits)
|
||||
- Explicit assertions
|
||||
- Self-cleaning tests
|
||||
```
|
||||
|
||||
#### Definition of Done Summary:
|
||||
|
||||
```markdown
|
||||
## Test Quality Checklist
|
||||
|
||||
✅ All tests pass on first run
|
||||
✅ No hard waits (waitForTimeout)
|
||||
✅ No conditionals for flow control
|
||||
✅ Assertions are explicit
|
||||
✅ Tests clean up after themselves
|
||||
✅ Tests can run in parallel
|
||||
✅ Execution time < 1.5 minutes per test
|
||||
✅ Test files < 300 lines
|
||||
```
|
||||
|
||||
### 7. Run the Tests
|
||||
|
||||
All tests should pass immediately since the feature exists:
|
||||
|
||||
**For Playwright:**
|
||||
```bash
|
||||
npx playwright test
|
||||
```
|
||||
|
||||
**For Cypress:**
|
||||
```bash
|
||||
npx cypress run
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
Running 15 tests using 4 workers
|
||||
|
||||
✓ tests/api/profile.spec.ts (4 tests) - 2.1s
|
||||
✓ tests/e2e/profile-workflow.spec.ts (2 tests) - 5.3s
|
||||
|
||||
15 passed (7.4s)
|
||||
```
|
||||
|
||||
**All green!** Tests pass because feature already exists.
|
||||
|
||||
### 8. Review Test Coverage
|
||||
|
||||
Check which scenarios are covered:
|
||||
|
||||
```bash
|
||||
# View test report
|
||||
npx playwright show-report
|
||||
|
||||
# Check coverage (if configured)
|
||||
npm run test:coverage
|
||||
```
|
||||
|
||||
Compare against:
|
||||
- Acceptance criteria from story
|
||||
- Test priorities from test design
|
||||
- Edge cases and error scenarios
|
||||
|
||||
## What You Get
|
||||
|
||||
### Comprehensive Test Suite
|
||||
- **API tests** - Fast, reliable backend testing
|
||||
- **E2E tests** - Critical user workflows
|
||||
- **Component tests** - UI component testing (if requested)
|
||||
- **Fixtures** - Shared utilities and setup
|
||||
|
||||
### Component Testing by Framework
|
||||
|
||||
TEA supports component testing using framework-appropriate tools:
|
||||
|
||||
| Your Framework | Component Testing Tool | Tests Location |
|
||||
| -------------- | ------------------------------ | ----------------------------------------- |
|
||||
| **Cypress** | Cypress Component Testing | `tests/component/` |
|
||||
| **Playwright** | Vitest + React Testing Library | `tests/component/` or `src/**/*.test.tsx` |
|
||||
|
||||
**Note:** Component tests use separate tooling from E2E tests:
|
||||
- Cypress users: TEA generates Cypress Component Tests
|
||||
- Playwright users: TEA generates Vitest + React Testing Library tests
|
||||
|
||||
### Quality Features
|
||||
- **Network-first patterns** - Wait for actual responses, not timeouts
|
||||
- **Deterministic tests** - No flakiness, no conditionals
|
||||
- **Self-cleaning** - Tests don't leave test data behind
|
||||
- **Parallel-safe** - Can run all tests concurrently
|
||||
|
||||
### Documentation
|
||||
- **Updated README** - How to run tests
|
||||
- **Test structure explanation** - Where tests live
|
||||
- **Definition of Done** - Quality standards
|
||||
|
||||
## Tips
|
||||
|
||||
### Start with Test Design
|
||||
|
||||
Run `*test-design` before `*automate` for better results:
|
||||
|
||||
```
|
||||
*test-design # Risk assessment, priorities
|
||||
*automate # Generate tests based on priorities
|
||||
```
|
||||
|
||||
TEA will focus on P0/P1 scenarios and skip low-value tests.
|
||||
|
||||
### Prioritize Test Levels
|
||||
|
||||
Not everything needs E2E tests:
|
||||
|
||||
**Good strategy:**
|
||||
```
|
||||
- P0 scenarios: API + E2E tests
|
||||
- P1 scenarios: API tests only
|
||||
- P2 scenarios: API tests (happy path)
|
||||
- P3 scenarios: Skip or add later
|
||||
```
|
||||
|
||||
**Why?**
|
||||
- API tests are 10x faster than E2E
|
||||
- API tests are more reliable (no browser flakiness)
|
||||
- E2E tests reserved for critical user journeys
|
||||
|
||||
### Avoid Duplicate Coverage
|
||||
|
||||
Tell TEA about existing tests:
|
||||
|
||||
```
|
||||
We already have tests in:
|
||||
- tests/e2e/profile-view.spec.ts (viewing profile)
|
||||
- tests/api/auth.spec.ts (authentication)
|
||||
|
||||
Don't duplicate that coverage
|
||||
```
|
||||
|
||||
TEA will analyze existing tests and only generate new scenarios.
|
||||
|
||||
### MCP Enhancements (Optional)
|
||||
|
||||
If you have MCP servers configured (`tea_use_mcp_enhancements: true`), TEA can use them during `*automate` for:
|
||||
|
||||
- **Healing mode:** Fix broken selectors, update assertions, enhance with trace analysis
|
||||
- **Recording mode:** Verify selectors with live browser, capture network requests
|
||||
|
||||
No prompts - TEA uses MCPs automatically when available. See [Enable MCP Enhancements](/docs/how-to/customization/enable-tea-mcp-enhancements.md) for setup.
|
||||
|
||||
### Generate Tests Incrementally
|
||||
|
||||
Don't generate all tests at once:
|
||||
|
||||
**Iteration 1:**
|
||||
```
|
||||
Generate P0 tests only (critical path)
|
||||
Run: *automate
|
||||
```
|
||||
|
||||
**Iteration 2:**
|
||||
```
|
||||
Generate P1 tests (high value scenarios)
|
||||
Run: *automate
|
||||
Tell TEA to avoid P0 coverage
|
||||
```
|
||||
|
||||
**Iteration 3:**
|
||||
```
|
||||
Generate P2 tests (if time permits)
|
||||
Run: *automate
|
||||
```
|
||||
|
||||
This iterative approach:
|
||||
- Provides fast feedback
|
||||
- Allows validation before proceeding
|
||||
- Keeps test generation focused
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Tests Pass But Coverage Is Incomplete
|
||||
|
||||
**Problem:** Tests pass but don't cover all scenarios.
|
||||
|
||||
**Cause:** TEA wasn't given complete context.
|
||||
|
||||
**Solution:** Provide more details:
|
||||
```
|
||||
Generate tests for:
|
||||
- All acceptance criteria in story-profile.md
|
||||
- Error scenarios (validation, authorization)
|
||||
- Edge cases (empty fields, long inputs)
|
||||
```
|
||||
|
||||
### Too Many Tests Generated
|
||||
|
||||
**Problem:** TEA generated 50 tests for a simple feature.
|
||||
|
||||
**Cause:** Didn't specify priorities or scope.
|
||||
|
||||
**Solution:** Be specific:
|
||||
```
|
||||
Generate ONLY:
|
||||
- P0 and P1 scenarios
|
||||
- API tests for all scenarios
|
||||
- E2E tests only for critical workflows
|
||||
- Skip P2/P3 for now
|
||||
```
|
||||
|
||||
### Tests Duplicate Existing Coverage
|
||||
|
||||
**Problem:** New tests cover the same scenarios as existing tests.
|
||||
|
||||
**Cause:** Didn't tell TEA about existing tests.
|
||||
|
||||
**Solution:** Specify existing coverage:
|
||||
```
|
||||
We already have these tests:
|
||||
- tests/api/profile.spec.ts (GET /api/profile)
|
||||
- tests/e2e/profile-view.spec.ts (viewing profile)
|
||||
|
||||
Generate tests for scenarios NOT covered by those files
|
||||
```
|
||||
|
||||
### MCP Enhancements for Better Selectors
|
||||
|
||||
If you have MCP servers configured, TEA verifies selectors against live browser. Otherwise, TEA generates accessible selectors (`getByRole`, `getByLabel`) by default.
|
||||
|
||||
Setup: Answer "Yes" to MCPs in BMad installer + configure MCP servers in your IDE. See [Enable MCP Enhancements](/docs/how-to/customization/enable-tea-mcp-enhancements.md).
|
||||
|
||||
## Related Guides
|
||||
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md) - Plan before generating
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Failing tests before implementation
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md) - Audit generated quality
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - **Why TEA generates quality tests** (foundational)
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Why prioritize P0 over P3
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - What makes tests good
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Reusable test patterns
|
||||
|
||||
## Reference
|
||||
|
||||
- [Command: *automate](/docs/reference/tea/commands.md#automate) - Full command reference
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - MCP and Playwright Utils options
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,679 @@
|
|||
---
|
||||
title: "How to Run NFR Assessment with TEA"
|
||||
description: Validate non-functional requirements for security, performance, reliability, and maintainability using TEA
|
||||
---
|
||||
|
||||
# How to Run NFR Assessment with TEA
|
||||
|
||||
Use TEA's `*nfr-assess` workflow to validate non-functional requirements (NFRs) with evidence-based assessment across security, performance, reliability, and maintainability.
|
||||
|
||||
## When to Use This
|
||||
|
||||
- Enterprise projects with compliance requirements
|
||||
- Projects with strict NFR thresholds
|
||||
- Before production release
|
||||
- When NFRs are critical to project success
|
||||
- Security or performance is mission-critical
|
||||
|
||||
**Best for:**
|
||||
- Enterprise track projects
|
||||
- Compliance-heavy industries (finance, healthcare, government)
|
||||
- High-traffic applications
|
||||
- Security-critical systems
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed
|
||||
- TEA agent available
|
||||
- NFRs defined in PRD or requirements doc
|
||||
- Evidence preferred but not required (test results, security scans, performance metrics)
|
||||
|
||||
**Note:** You can run NFR assessment without complete evidence. TEA will mark categories as CONCERNS where evidence is missing and document what's needed.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Run the NFR Assessment Workflow
|
||||
|
||||
Start a fresh chat and run:
|
||||
|
||||
```
|
||||
*nfr-assess
|
||||
```
|
||||
|
||||
This loads TEA and starts the NFR assessment workflow.
|
||||
|
||||
### 2. Specify NFR Categories
|
||||
|
||||
TEA will ask which NFR categories to assess.
|
||||
|
||||
**Available Categories:**
|
||||
|
||||
| Category | Focus Areas |
|
||||
|----------|-------------|
|
||||
| **Security** | Authentication, authorization, encryption, vulnerabilities, security headers, input validation |
|
||||
| **Performance** | Response time, throughput, resource usage, database queries, frontend load time |
|
||||
| **Reliability** | Error handling, recovery mechanisms, availability, failover, data backup |
|
||||
| **Maintainability** | Code quality, test coverage, technical debt, documentation, dependency health |
|
||||
|
||||
**Example Response:**
|
||||
```
|
||||
Assess:
|
||||
- Security (critical for user data)
|
||||
- Performance (API must be fast)
|
||||
- Reliability (99.9% uptime requirement)
|
||||
|
||||
Skip maintainability for now
|
||||
```
|
||||
|
||||
### 3. Provide NFR Thresholds
|
||||
|
||||
TEA will ask for specific thresholds for each category.
|
||||
|
||||
**Critical Principle: Never guess thresholds.**
|
||||
|
||||
If you don't know the exact requirement, tell TEA to mark as CONCERNS and request clarification from stakeholders.
|
||||
|
||||
#### Security Thresholds
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Requirements:
|
||||
- All endpoints require authentication: YES
|
||||
- Data encrypted at rest: YES (PostgreSQL TDE)
|
||||
- Zero critical vulnerabilities: YES (npm audit)
|
||||
- Input validation on all endpoints: YES (Zod schemas)
|
||||
- Security headers configured: YES (helmet.js)
|
||||
```
|
||||
|
||||
#### Performance Thresholds
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Requirements:
|
||||
- API response time P99: < 200ms
|
||||
- API response time P95: < 150ms
|
||||
- Throughput: > 1000 requests/second
|
||||
- Frontend initial load: < 2 seconds
|
||||
- Database query time P99: < 50ms
|
||||
```
|
||||
|
||||
#### Reliability Thresholds
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Requirements:
|
||||
- Error handling: All endpoints return structured errors
|
||||
- Availability: 99.9% uptime
|
||||
- Recovery time: < 5 minutes (RTO)
|
||||
- Data backup: Daily automated backups
|
||||
- Failover: Automatic with < 30s downtime
|
||||
```
|
||||
|
||||
#### Maintainability Thresholds
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Requirements:
|
||||
- Test coverage: > 80%
|
||||
- Code quality: SonarQube grade A
|
||||
- Documentation: All APIs documented
|
||||
- Dependency age: < 6 months outdated
|
||||
- Technical debt: < 10% of codebase
|
||||
```
|
||||
|
||||
### 4. Provide Evidence
|
||||
|
||||
TEA will ask where to find evidence for each requirement.
|
||||
|
||||
**Evidence Sources:**
|
||||
|
||||
| Category | Evidence Type | Location |
|
||||
|----------|---------------|----------|
|
||||
| Security | Security scan reports | `/reports/security-scan.pdf` |
|
||||
| Security | Vulnerability scan | `npm audit`, `snyk test` results |
|
||||
| Security | Auth test results | Test reports showing auth coverage |
|
||||
| Performance | Load test results | `/reports/k6-load-test.json` |
|
||||
| Performance | APM data | Datadog, New Relic dashboards |
|
||||
| Performance | Lighthouse scores | `/reports/lighthouse.json` |
|
||||
| Reliability | Error rate metrics | Production monitoring dashboards |
|
||||
| Reliability | Uptime data | StatusPage, PagerDuty logs |
|
||||
| Maintainability | Coverage reports | `/reports/coverage/index.html` |
|
||||
| Maintainability | Code quality | SonarQube dashboard |
|
||||
|
||||
**Example Response:**
|
||||
```
|
||||
Evidence:
|
||||
- Security: npm audit results (clean), auth tests 15/15 passing
|
||||
- Performance: k6 load test at /reports/k6-results.json
|
||||
- Reliability: Error rate 0.01% in staging (logs in Datadog)
|
||||
|
||||
Don't have:
|
||||
- Uptime data (new system, no baseline)
|
||||
- Mark as CONCERNS and request monitoring setup
|
||||
```
|
||||
|
||||
### 5. Review NFR Assessment Report
|
||||
|
||||
TEA generates a comprehensive assessment report.
|
||||
|
||||
#### Assessment Report (`nfr-assessment.md`):
|
||||
|
||||
```markdown
|
||||
# Non-Functional Requirements Assessment
|
||||
|
||||
**Date:** 2026-01-13
|
||||
**Epic:** User Profile Management
|
||||
**Release:** v1.2.0
|
||||
**Overall Decision:** CONCERNS ⚠️
|
||||
|
||||
## Executive Summary
|
||||
|
||||
| Category | Status | Critical Issues |
|
||||
|----------|--------|-----------------|
|
||||
| Security | PASS ✅ | 0 |
|
||||
| Performance | CONCERNS ⚠️ | 2 |
|
||||
| Reliability | PASS ✅ | 0 |
|
||||
| Maintainability | PASS ✅ | 0 |
|
||||
|
||||
**Decision Rationale:**
|
||||
Performance metrics below target (P99 latency, throughput). Mitigation plan in place. Security and reliability meet all requirements.
|
||||
|
||||
---
|
||||
|
||||
## Security Assessment
|
||||
|
||||
**Status:** PASS ✅
|
||||
|
||||
### Requirements Met
|
||||
|
||||
| Requirement | Target | Actual | Status |
|
||||
|-------------|--------|--------|--------|
|
||||
| Authentication required | All endpoints | 100% enforced | ✅ |
|
||||
| Data encryption at rest | PostgreSQL TDE | Enabled | ✅ |
|
||||
| Critical vulnerabilities | 0 | 0 | ✅ |
|
||||
| Input validation | All endpoints | Zod schemas on 100% | ✅ |
|
||||
| Security headers | Configured | helmet.js enabled | ✅ |
|
||||
|
||||
### Evidence
|
||||
|
||||
**Security Scan:**
|
||||
```bash
|
||||
$ npm audit
|
||||
found 0 vulnerabilities
|
||||
```
|
||||
|
||||
**Authentication Tests:**
|
||||
- 15/15 auth tests passing
|
||||
- Tested unauthorized access (401 responses)
|
||||
- Token validation working
|
||||
|
||||
**Penetration Testing:**
|
||||
- Report: `/reports/pentest-2026-01.pdf`
|
||||
- Findings: 0 critical, 2 low (addressed)
|
||||
|
||||
**Conclusion:** All security requirements met. No blockers.
|
||||
|
||||
---
|
||||
|
||||
## Performance Assessment
|
||||
|
||||
**Status:** CONCERNS ⚠️
|
||||
|
||||
### Requirements Status
|
||||
|
||||
| Metric | Target | Actual | Status |
|
||||
|--------|--------|--------|--------|
|
||||
| API response P99 | < 200ms | 350ms | ❌ Exceeds |
|
||||
| API response P95 | < 150ms | 180ms | ⚠️ Exceeds |
|
||||
| Throughput | > 1000 rps | 850 rps | ⚠️ Below |
|
||||
| Frontend load | < 2s | 1.8s | ✅ Met |
|
||||
| DB query P99 | < 50ms | 85ms | ❌ Exceeds |
|
||||
|
||||
### Issues Identified
|
||||
|
||||
#### Issue 1: P99 Latency Exceeds Target
|
||||
|
||||
**Measured:** 350ms P99 (target: <200ms)
|
||||
**Root Cause:** Database queries not optimized
|
||||
- Missing indexes on profile queries
|
||||
- N+1 query problem in profile endpoint
|
||||
|
||||
**Impact:** User experience degraded for 1% of requests
|
||||
|
||||
**Mitigation Plan:**
|
||||
- Add composite index on `(user_id, profile_id)` - backend team, 2 days
|
||||
- Refactor profile endpoint to use joins instead of multiple queries - backend team, 3 days
|
||||
- Re-run load tests after optimization - QA team, 1 day
|
||||
|
||||
**Owner:** Backend team lead
|
||||
**Deadline:** Before release (January 20, 2026)
|
||||
|
||||
#### Issue 2: Throughput Below Target
|
||||
|
||||
**Measured:** 850 rps (target: >1000 rps)
|
||||
**Root Cause:** Connection pool size too small
|
||||
- PostgreSQL max_connections = 100 (too low)
|
||||
- No connection pooling in application
|
||||
|
||||
**Impact:** System cannot handle expected traffic
|
||||
|
||||
**Mitigation Plan:**
|
||||
- Increase PostgreSQL max_connections to 500 - DevOps, 1 day
|
||||
- Implement connection pooling with pg-pool - backend team, 2 days
|
||||
- Re-run load tests - QA team, 1 day
|
||||
|
||||
**Owner:** DevOps + Backend team
|
||||
**Deadline:** Before release (January 20, 2026)
|
||||
|
||||
### Evidence
|
||||
|
||||
**Load Testing:**
|
||||
```
|
||||
Tool: k6
|
||||
Duration: 10 minutes
|
||||
Virtual Users: 500 concurrent
|
||||
Report: /reports/k6-load-test.json
|
||||
```
|
||||
|
||||
**Results:**
|
||||
```
|
||||
scenarios: (100.00%) 1 scenario, 500 max VUs, 10m30s max duration
|
||||
✓ http_req_duration..............: avg=250ms min=45ms med=180ms max=2.1s p(90)=280ms p(95)=350ms
|
||||
http_reqs......................: 85000 (850/s)
|
||||
http_req_failed................: 0.1%
|
||||
```
|
||||
|
||||
**APM Data:**
|
||||
- Tool: Datadog
|
||||
- Dashboard: <https://app.datadoghq.com/dashboard/abc123>
|
||||
|
||||
**Conclusion:** Performance issues identified with mitigation plan. Re-assess after optimization.
|
||||
|
||||
---
|
||||
|
||||
## Reliability Assessment
|
||||
|
||||
**Status:** PASS ✅
|
||||
|
||||
### Requirements Met
|
||||
|
||||
| Requirement | Target | Actual | Status |
|
||||
|-------------|--------|--------|--------|
|
||||
| Error handling | Structured errors | 100% endpoints | ✅ |
|
||||
| Availability | 99.9% uptime | 99.95% (staging) | ✅ |
|
||||
| Recovery time | < 5 min (RTO) | 3 min (tested) | ✅ |
|
||||
| Data backup | Daily | Automated daily | ✅ |
|
||||
| Failover | < 30s downtime | 15s (tested) | ✅ |
|
||||
|
||||
### Evidence
|
||||
|
||||
**Error Handling Tests:**
|
||||
- All endpoints return structured JSON errors
|
||||
- Error codes standardized (400, 401, 403, 404, 500)
|
||||
- Error messages user-friendly (no stack traces)
|
||||
|
||||
**Chaos Engineering:**
|
||||
- Tested database failover: 15s downtime ✅
|
||||
- Tested service crash recovery: 3 min ✅
|
||||
- Tested network partition: Graceful degradation ✅
|
||||
|
||||
**Monitoring:**
|
||||
- Staging uptime (30 days): 99.95%
|
||||
- Error rate: 0.01% (target: <0.1%)
|
||||
- P50 availability: 100%
|
||||
|
||||
**Conclusion:** All reliability requirements exceeded. No issues.
|
||||
|
||||
---
|
||||
|
||||
## Maintainability Assessment
|
||||
|
||||
**Status:** PASS ✅
|
||||
|
||||
### Requirements Met
|
||||
|
||||
| Requirement | Target | Actual | Status |
|
||||
|-------------|--------|--------|--------|
|
||||
| Test coverage | > 80% | 85% | ✅ |
|
||||
| Code quality | Grade A | Grade A | ✅ |
|
||||
| Documentation | All APIs | 100% documented | ✅ |
|
||||
| Outdated dependencies | < 6 months | 3 months avg | ✅ |
|
||||
| Technical debt | < 10% | 7% | ✅ |
|
||||
|
||||
### Evidence
|
||||
|
||||
**Test Coverage:**
|
||||
```
|
||||
Statements : 85.2% ( 1205/1414 )
|
||||
Branches : 82.1% ( 412/502 )
|
||||
Functions : 88.5% ( 201/227 )
|
||||
Lines : 85.2% ( 1205/1414 )
|
||||
```
|
||||
|
||||
**Code Quality:**
|
||||
- SonarQube: Grade A
|
||||
- Maintainability rating: A
|
||||
- Technical debt ratio: 7%
|
||||
- Code smells: 12 (all minor)
|
||||
|
||||
**Documentation:**
|
||||
- API docs: 100% coverage (OpenAPI spec)
|
||||
- README: Complete and up-to-date
|
||||
- Architecture docs: ADRs for all major decisions
|
||||
|
||||
**Conclusion:** All maintainability requirements met. Codebase is healthy.
|
||||
|
||||
---
|
||||
|
||||
## Overall Gate Decision
|
||||
|
||||
### Decision: CONCERNS ⚠️
|
||||
|
||||
**Rationale:**
|
||||
- **Blockers:** None
|
||||
- **Concerns:** Performance metrics below target (P99 latency, throughput)
|
||||
- **Mitigation:** Plan in place with clear owners and deadlines (5 days total)
|
||||
- **Passing:** Security, reliability, maintainability all green
|
||||
|
||||
### Actions Required Before Release
|
||||
|
||||
1. **Optimize database queries** (backend team, 3 days)
|
||||
- Add indexes
|
||||
- Fix N+1 queries
|
||||
- Implement connection pooling
|
||||
|
||||
2. **Re-run performance tests** (QA team, 1 day)
|
||||
- Validate P99 < 200ms
|
||||
- Validate throughput > 1000 rps
|
||||
|
||||
3. **Update this assessment** (TEA, 1 hour)
|
||||
- Re-run `*nfr-assess` with new results
|
||||
- Confirm PASS status
|
||||
|
||||
### Waiver Option (If Business Approves)
|
||||
|
||||
If business decides to deploy with current performance:
|
||||
|
||||
**Waiver Justification:**
|
||||
```markdown
|
||||
## Performance Waiver
|
||||
|
||||
**Waived By:** VP Engineering, Product Manager
|
||||
**Date:** 2026-01-15
|
||||
**Reason:** Business priority to launch by Q1
|
||||
**Conditions:**
|
||||
- Set monitoring alerts for P99 > 300ms
|
||||
- Plan optimization for v1.3 (February release)
|
||||
- Document known performance limitations in release notes
|
||||
|
||||
**Accepted Risk:**
|
||||
- 1% of users experience slower response (350ms vs 200ms)
|
||||
- System can handle current traffic (850 rps sufficient for launch)
|
||||
- Optimization planned for next release
|
||||
```
|
||||
|
||||
### Approvals
|
||||
|
||||
- [ ] Product Manager - Review business impact
|
||||
- [ ] Tech Lead - Review mitigation plan
|
||||
- [ ] QA Lead - Validate test evidence
|
||||
- [ ] DevOps - Confirm infrastructure ready
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Plan Post-Release
|
||||
|
||||
**Performance Alerts:**
|
||||
- P99 latency > 400ms (critical)
|
||||
- Throughput < 700 rps (warning)
|
||||
- Error rate > 1% (critical)
|
||||
|
||||
**Review Cadence:**
|
||||
- Daily: Check performance dashboards
|
||||
- Weekly: Review alert trends
|
||||
- Monthly: Re-assess NFRs
|
||||
```
|
||||
|
||||
## What You Get
|
||||
|
||||
### NFR Assessment Report
|
||||
- Category-by-category analysis (Security, Performance, Reliability, Maintainability)
|
||||
- Requirements status (target vs actual)
|
||||
- Evidence for each requirement
|
||||
- Issues identified with root cause analysis
|
||||
|
||||
### Gate Decision
|
||||
- **PASS** ✅ - All NFRs met, ready to release
|
||||
- **CONCERNS** ⚠️ - Some NFRs not met, mitigation plan exists
|
||||
- **FAIL** ❌ - Critical NFRs not met, blocks release
|
||||
- **WAIVED** ⏭️ - Business-approved waiver with documented risk
|
||||
|
||||
### Mitigation Plans
|
||||
- Specific actions to address concerns
|
||||
- Owners and deadlines
|
||||
- Re-assessment criteria
|
||||
|
||||
### Monitoring Plan
|
||||
- Post-release monitoring strategy
|
||||
- Alert thresholds
|
||||
- Review cadence
|
||||
|
||||
## Tips
|
||||
|
||||
### Run NFR Assessment Early
|
||||
|
||||
**Phase 2 (Enterprise):**
|
||||
Run `*nfr-assess` during planning to:
|
||||
- Identify NFR requirements early
|
||||
- Plan for performance testing
|
||||
- Budget for security audits
|
||||
- Set up monitoring infrastructure
|
||||
|
||||
**Phase 4 or Gate:**
|
||||
Re-run before release to validate all requirements met.
|
||||
|
||||
### Never Guess Thresholds
|
||||
|
||||
If you don't know the NFR target:
|
||||
|
||||
**Don't:**
|
||||
```
|
||||
API response time should probably be under 500ms
|
||||
```
|
||||
|
||||
**Do:**
|
||||
```
|
||||
Mark as CONCERNS - Request threshold from stakeholders
|
||||
"What is the acceptable API response time?"
|
||||
```
|
||||
|
||||
### Collect Evidence Beforehand
|
||||
|
||||
Before running `*nfr-assess`, gather:
|
||||
|
||||
**Security:**
|
||||
```bash
|
||||
npm audit # Vulnerability scan
|
||||
snyk test # Alternative security scan
|
||||
npm run test:security # Security test suite
|
||||
```
|
||||
|
||||
**Performance:**
|
||||
```bash
|
||||
npm run test:load # k6 or artillery load tests
|
||||
npm run test:lighthouse # Frontend performance
|
||||
npm run test:db-performance # Database query analysis
|
||||
```
|
||||
|
||||
**Reliability:**
|
||||
- Production error rate (last 30 days)
|
||||
- Uptime data (StatusPage, PagerDuty)
|
||||
- Incident response times
|
||||
|
||||
**Maintainability:**
|
||||
```bash
|
||||
npm run test:coverage # Test coverage report
|
||||
npm run lint # Code quality check
|
||||
npm outdated # Dependency freshness
|
||||
```
|
||||
|
||||
### Use Real Data, Not Assumptions
|
||||
|
||||
**Don't:**
|
||||
```
|
||||
System is probably fast enough
|
||||
Security seems fine
|
||||
```
|
||||
|
||||
**Do:**
|
||||
```
|
||||
Load test results show P99 = 350ms
|
||||
npm audit shows 0 vulnerabilities
|
||||
Test coverage report shows 85%
|
||||
```
|
||||
|
||||
Evidence-based decisions prevent surprises in production.
|
||||
|
||||
### Document Waivers Thoroughly
|
||||
|
||||
If business approves waiver:
|
||||
|
||||
**Required:**
|
||||
- Who approved (name, role, date)
|
||||
- Why (business justification)
|
||||
- Conditions (monitoring, future plans)
|
||||
- Accepted risk (quantified impact)
|
||||
|
||||
**Example:**
|
||||
```markdown
|
||||
Waived by: CTO, VP Product (2026-01-15)
|
||||
Reason: Q1 launch critical for investor demo
|
||||
Conditions: Optimize in v1.3, monitor closely
|
||||
Risk: 1% of users experience 350ms latency (acceptable for launch)
|
||||
```
|
||||
|
||||
### Re-Assess After Fixes
|
||||
|
||||
After implementing mitigations:
|
||||
|
||||
```
|
||||
1. Fix performance issues
|
||||
2. Run load tests again
|
||||
3. Run *nfr-assess with new evidence
|
||||
4. Verify PASS status
|
||||
```
|
||||
|
||||
Don't deploy with CONCERNS without mitigation or waiver.
|
||||
|
||||
### Integrate with Release Checklist
|
||||
|
||||
```markdown
|
||||
## Release Checklist
|
||||
|
||||
### Pre-Release
|
||||
- [ ] All tests passing
|
||||
- [ ] Test coverage > 80%
|
||||
- [ ] Run *nfr-assess
|
||||
- [ ] NFR status: PASS or WAIVED
|
||||
|
||||
### Performance
|
||||
- [ ] Load tests completed
|
||||
- [ ] P99 latency meets threshold
|
||||
- [ ] Throughput meets threshold
|
||||
|
||||
### Security
|
||||
- [ ] Security scan clean
|
||||
- [ ] Auth tests passing
|
||||
- [ ] Penetration test complete
|
||||
|
||||
### Post-Release
|
||||
- [ ] Monitoring alerts configured
|
||||
- [ ] Dashboards updated
|
||||
- [ ] Incident response plan ready
|
||||
```
|
||||
|
||||
## Common Issues
|
||||
|
||||
### No Evidence Available
|
||||
|
||||
**Problem:** Don't have performance data, security scans, etc.
|
||||
|
||||
**Solution:**
|
||||
```
|
||||
Mark as CONCERNS for categories without evidence
|
||||
Document what evidence is needed
|
||||
Set up tests/scans before re-assessment
|
||||
```
|
||||
|
||||
**Don't block on missing evidence** - document what's needed and proceed.
|
||||
|
||||
### Thresholds Too Strict
|
||||
|
||||
**Problem:** Can't meet unrealistic thresholds.
|
||||
|
||||
**Symptoms:**
|
||||
- P99 < 50ms (impossible for complex queries)
|
||||
- 100% test coverage (impractical)
|
||||
- Zero technical debt (unrealistic)
|
||||
|
||||
**Solution:**
|
||||
```
|
||||
Negotiate thresholds with stakeholders:
|
||||
- "P99 < 50ms is unrealistic for our DB queries"
|
||||
- "Propose P99 < 200ms based on industry standards"
|
||||
- "Show evidence from load tests"
|
||||
```
|
||||
|
||||
Use data to negotiate realistic requirements.
|
||||
|
||||
### Assessment Takes Too Long
|
||||
|
||||
**Problem:** Gathering evidence for all categories is time-consuming.
|
||||
|
||||
**Solution:** Focus on critical categories first:
|
||||
|
||||
**For most projects:**
|
||||
```
|
||||
Priority 1: Security (always critical)
|
||||
Priority 2: Performance (if high-traffic)
|
||||
Priority 3: Reliability (if uptime critical)
|
||||
Priority 4: Maintainability (nice to have)
|
||||
```
|
||||
|
||||
Assess categories incrementally, not all at once.
|
||||
|
||||
### CONCERNS vs FAIL - When to Block?
|
||||
|
||||
**CONCERNS** ⚠️:
|
||||
- Issues exist but not critical
|
||||
- Mitigation plan in place
|
||||
- Business accepts risk (with waiver)
|
||||
- Can deploy with monitoring
|
||||
|
||||
**FAIL** ❌:
|
||||
- Critical security vulnerability (CVE critical)
|
||||
- System unusable (error rate >10%)
|
||||
- Data loss risk (no backups)
|
||||
- Zero mitigation possible
|
||||
|
||||
**Rule of thumb:** If you can mitigate or monitor, use CONCERNS. Reserve FAIL for absolute blockers.
|
||||
|
||||
## Related Guides
|
||||
|
||||
- [How to Run Trace](/docs/how-to/workflows/run-trace.md) - Gate decision complements NFR
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md) - Quality complements NFR
|
||||
- [Run TEA for Enterprise](/docs/how-to/brownfield/use-tea-for-enterprise.md) - Enterprise workflow
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Risk assessment principles
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - NFR in release gates
|
||||
|
||||
## Reference
|
||||
|
||||
- [Command: *nfr-assess](/docs/reference/tea/commands.md#nfr-assess) - Full command reference
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Enterprise config options
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
title: "How to Run Test Design"
|
||||
title: "How to Run Test Design with TEA"
|
||||
description: How to create comprehensive test plans using TEA's test-design workflow
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,605 @@
|
|||
---
|
||||
title: "How to Run Test Review with TEA"
|
||||
description: Audit test quality using TEA's comprehensive knowledge base and get 0-100 scoring
|
||||
---
|
||||
|
||||
# How to Run Test Review with TEA
|
||||
|
||||
Use TEA's `*test-review` workflow to audit test quality with objective scoring and actionable feedback. TEA reviews tests against its knowledge base of best practices.
|
||||
|
||||
## When to Use This
|
||||
|
||||
- Want to validate test quality objectively
|
||||
- Need quality metrics for release gates
|
||||
- Preparing for production deployment
|
||||
- Reviewing team-written tests
|
||||
- Auditing AI-generated tests
|
||||
- Onboarding new team members (show good patterns)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed
|
||||
- TEA agent available
|
||||
- Tests written (to review)
|
||||
- Test framework configured
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Load TEA Agent
|
||||
|
||||
Start a fresh chat and load TEA:
|
||||
|
||||
```
|
||||
*tea
|
||||
```
|
||||
|
||||
### 2. Run the Test Review Workflow
|
||||
|
||||
```
|
||||
*test-review
|
||||
```
|
||||
|
||||
### 3. Specify Review Scope
|
||||
|
||||
TEA will ask what to review.
|
||||
|
||||
#### Option A: Single File
|
||||
|
||||
Review one test file:
|
||||
|
||||
```
|
||||
tests/e2e/checkout.spec.ts
|
||||
```
|
||||
|
||||
**Best for:**
|
||||
- Reviewing specific failing tests
|
||||
- Quick feedback on new tests
|
||||
- Learning from specific examples
|
||||
|
||||
#### Option B: Directory
|
||||
|
||||
Review all tests in a directory:
|
||||
|
||||
```
|
||||
tests/e2e/
|
||||
```
|
||||
|
||||
**Best for:**
|
||||
- Reviewing E2E test suite
|
||||
- Comparing test quality across files
|
||||
- Finding patterns of issues
|
||||
|
||||
#### Option C: Entire Suite
|
||||
|
||||
Review all tests:
|
||||
|
||||
```
|
||||
tests/
|
||||
```
|
||||
|
||||
**Best for:**
|
||||
- Release gate quality check
|
||||
- Comprehensive audit
|
||||
- Establishing baseline metrics
|
||||
|
||||
### 4. Review the Quality Report
|
||||
|
||||
TEA generates a comprehensive quality report with scoring.
|
||||
|
||||
#### Report Structure (`test-review.md`):
|
||||
|
||||
```markdown
|
||||
# Test Quality Review Report
|
||||
|
||||
**Date:** 2026-01-13
|
||||
**Scope:** tests/e2e/
|
||||
**Overall Score:** 76/100
|
||||
|
||||
## Summary
|
||||
|
||||
- **Tests Reviewed:** 12
|
||||
- **Passing Quality:** 9 tests (75%)
|
||||
- **Needs Improvement:** 3 tests (25%)
|
||||
- **Critical Issues:** 2
|
||||
- **Recommendations:** 6
|
||||
|
||||
## Critical Issues
|
||||
|
||||
### 1. Hard Waits Detected
|
||||
|
||||
**File:** `tests/e2e/checkout.spec.ts:45`
|
||||
**Issue:** Using `page.waitForTimeout(3000)`
|
||||
**Impact:** Test is flaky and unnecessarily slow
|
||||
**Severity:** Critical
|
||||
|
||||
**Current Code:**
|
||||
```typescript
|
||||
await page.click('button[type="submit"]');
|
||||
await page.waitForTimeout(3000); // ❌ Hard wait
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
```
|
||||
|
||||
**Fix:**
|
||||
```typescript
|
||||
await page.click('button[type="submit"]');
|
||||
// Wait for the API response that triggers success message
|
||||
await page.waitForResponse(resp =>
|
||||
resp.url().includes('/api/checkout') && resp.ok()
|
||||
);
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
```
|
||||
|
||||
**Why This Matters:**
|
||||
- Hard waits are fixed timeouts that don't wait for actual conditions
|
||||
- Tests fail intermittently on slower machines
|
||||
- Wastes time waiting even when response is fast
|
||||
- Network-first patterns are more reliable
|
||||
|
||||
---
|
||||
|
||||
### 2. Conditional Flow Control
|
||||
|
||||
**File:** `tests/e2e/profile.spec.ts:28`
|
||||
**Issue:** Using if/else to handle optional elements
|
||||
**Impact:** Non-deterministic test behavior
|
||||
**Severity:** Critical
|
||||
|
||||
**Current Code:**
|
||||
```typescript
|
||||
if (await page.locator('.banner').isVisible()) {
|
||||
await page.click('.dismiss');
|
||||
}
|
||||
// ❌ Test behavior changes based on banner presence
|
||||
```
|
||||
|
||||
**Fix:**
|
||||
```typescript
|
||||
// Option 1: Make banner presence deterministic
|
||||
await expect(page.locator('.banner')).toBeVisible();
|
||||
await page.click('.dismiss');
|
||||
|
||||
// Option 2: Test both scenarios separately
|
||||
test('should show banner for new users', async ({ page }) => {
|
||||
// Test with banner
|
||||
});
|
||||
|
||||
test('should not show banner for returning users', async ({ page }) => {
|
||||
// Test without banner
|
||||
});
|
||||
```
|
||||
|
||||
**Why This Matters:**
|
||||
- Tests should be deterministic (same result every run)
|
||||
- Conditionals hide bugs (what if banner should always show?)
|
||||
- Makes debugging harder
|
||||
- Violates test isolation principle
|
||||
|
||||
## Recommendations
|
||||
|
||||
### 1. Extract Repeated Setup
|
||||
|
||||
**File:** `tests/e2e/profile.spec.ts`
|
||||
**Issue:** Login code duplicated in every test
|
||||
**Severity:** Medium
|
||||
**Impact:** Maintenance burden, test verbosity
|
||||
|
||||
**Current:**
|
||||
```typescript
|
||||
test('test 1', async ({ page }) => {
|
||||
await page.goto('/login');
|
||||
await page.fill('[name="email"]', 'test@example.com');
|
||||
await page.fill('[name="password"]', 'password');
|
||||
await page.click('button[type="submit"]');
|
||||
// Test logic...
|
||||
});
|
||||
|
||||
test('test 2', async ({ page }) => {
|
||||
// Same login code repeated
|
||||
});
|
||||
```
|
||||
|
||||
**Fix (Vanilla Playwright):**
|
||||
```typescript
|
||||
// Create fixture in tests/support/fixtures/auth.ts
|
||||
import { test as base, Page } from '@playwright/test';
|
||||
|
||||
export const test = base.extend<{ authenticatedPage: Page }>({
|
||||
authenticatedPage: async ({ page }, use) => {
|
||||
await page.goto('/login');
|
||||
await page.getByLabel('Email').fill('test@example.com');
|
||||
await page.getByLabel('Password').fill('password');
|
||||
await page.getByRole('button', { name: 'Sign in' }).click();
|
||||
await page.waitForURL(/\/dashboard/);
|
||||
await use(page);
|
||||
}
|
||||
});
|
||||
|
||||
// Use in tests
|
||||
test('test 1', async ({ authenticatedPage }) => {
|
||||
// Already logged in
|
||||
});
|
||||
```
|
||||
|
||||
**Better (With Playwright Utils):**
|
||||
```typescript
|
||||
// Use built-in auth-session fixture
|
||||
import { test as base } from '@playwright/test';
|
||||
import { createAuthFixtures } from '@seontechnologies/playwright-utils/auth-session';
|
||||
|
||||
export const test = base.extend(createAuthFixtures());
|
||||
|
||||
// Use in tests - even simpler
|
||||
test('test 1', async ({ page, authToken }) => {
|
||||
// authToken already available (persisted, reused)
|
||||
await page.goto('/dashboard');
|
||||
// Already authenticated via authToken
|
||||
});
|
||||
```
|
||||
|
||||
**Playwright Utils Benefits:**
|
||||
- Token persisted to disk (faster subsequent runs)
|
||||
- Multi-user support out of the box
|
||||
- Automatic token renewal if expired
|
||||
- No manual login flow needed
|
||||
|
||||
---
|
||||
|
||||
### 2. Add Network Assertions
|
||||
|
||||
**File:** `tests/e2e/api-calls.spec.ts`
|
||||
**Issue:** No verification of API responses
|
||||
**Severity:** Low
|
||||
**Impact:** Tests don't catch API errors
|
||||
|
||||
**Current:**
|
||||
```typescript
|
||||
await page.click('button[name="save"]');
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
// ❌ What if API returned 500 but UI shows cached success?
|
||||
```
|
||||
|
||||
**Enhancement:**
|
||||
```typescript
|
||||
const responsePromise = page.waitForResponse(
|
||||
resp => resp.url().includes('/api/profile') && resp.status() === 200
|
||||
);
|
||||
await page.click('button[name="save"]');
|
||||
const response = await responsePromise;
|
||||
|
||||
// Verify API response
|
||||
const data = await response.json();
|
||||
expect(data.success).toBe(true);
|
||||
|
||||
// Verify UI
|
||||
await expect(page.locator('.success')).toBeVisible();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Improve Test Names
|
||||
|
||||
**File:** `tests/e2e/checkout.spec.ts`
|
||||
**Issue:** Vague test names
|
||||
**Severity:** Low
|
||||
**Impact:** Hard to understand test purpose
|
||||
|
||||
**Current:**
|
||||
```typescript
|
||||
test('should work', async ({ page }) => { });
|
||||
test('test checkout', async ({ page }) => { });
|
||||
```
|
||||
|
||||
**Better:**
|
||||
```typescript
|
||||
test('should complete checkout with valid credit card', async ({ page }) => { });
|
||||
test('should show validation error for expired card', async ({ page }) => { });
|
||||
```
|
||||
|
||||
## Quality Scores by Category
|
||||
|
||||
| Category | Score | Target | Status |
|
||||
|----------|-------|--------|--------|
|
||||
| **Determinism** | 26/35 | 30/35 | ⚠️ Needs Improvement |
|
||||
| **Isolation** | 22/25 | 20/25 | ✅ Good |
|
||||
| **Assertions** | 18/20 | 16/20 | ✅ Good |
|
||||
| **Structure** | 7/10 | 8/10 | ⚠️ Minor Issues |
|
||||
| **Performance** | 3/10 | 8/10 | ❌ Critical |
|
||||
|
||||
### Scoring Breakdown
|
||||
|
||||
**Determinism (35 points max):**
|
||||
- No hard waits: 0/10 ❌ (found 3 instances)
|
||||
- No conditionals: 8/10 ⚠️ (found 2 instances)
|
||||
- No try-catch flow control: 10/10 ✅
|
||||
- Network-first patterns: 8/15 ⚠️ (some tests missing)
|
||||
|
||||
**Isolation (25 points max):**
|
||||
- Self-cleaning: 20/20 ✅
|
||||
- No global state: 5/5 ✅
|
||||
- Parallel-safe: 0/0 ✅ (not tested)
|
||||
|
||||
**Assertions (20 points max):**
|
||||
- Explicit in test body: 15/15 ✅
|
||||
- Specific and meaningful: 3/5 ⚠️ (some weak assertions)
|
||||
|
||||
**Structure (10 points max):**
|
||||
- Test size < 300 lines: 5/5 ✅
|
||||
- Clear names: 2/5 ⚠️ (some vague names)
|
||||
|
||||
**Performance (10 points max):**
|
||||
- Execution time < 1.5 min: 3/10 ❌ (3 tests exceed limit)
|
||||
|
||||
## Files Reviewed
|
||||
|
||||
| File | Score | Issues | Status |
|
||||
|------|-------|--------|--------|
|
||||
| `tests/e2e/checkout.spec.ts` | 65/100 | 4 | ❌ Needs Work |
|
||||
| `tests/e2e/profile.spec.ts` | 72/100 | 3 | ⚠️ Needs Improvement |
|
||||
| `tests/e2e/search.spec.ts` | 88/100 | 1 | ✅ Good |
|
||||
| `tests/api/profile.spec.ts` | 92/100 | 0 | ✅ Excellent |
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Immediate (Fix Critical Issues)
|
||||
1. Remove hard waits in `checkout.spec.ts` (line 45, 67, 89)
|
||||
2. Fix conditional in `profile.spec.ts` (line 28)
|
||||
3. Optimize slow tests in `checkout.spec.ts`
|
||||
|
||||
### Short-term (Apply Recommendations)
|
||||
4. Extract login fixture from `profile.spec.ts`
|
||||
5. Add network assertions to `api-calls.spec.ts`
|
||||
6. Improve test names in `checkout.spec.ts`
|
||||
|
||||
### Long-term (Continuous Improvement)
|
||||
7. Re-run `*test-review` after fixes (target: 85/100)
|
||||
8. Add performance budgets to CI
|
||||
9. Document test patterns for team
|
||||
|
||||
## Knowledge Base References
|
||||
|
||||
TEA reviewed against these patterns:
|
||||
- [test-quality.md](/docs/reference/tea/knowledge-base.md#test-quality) - Execution limits, isolation
|
||||
- [network-first.md](/docs/reference/tea/knowledge-base.md#network-first) - Deterministic waits
|
||||
- [timing-debugging.md](/docs/reference/tea/knowledge-base.md#timing-debugging) - Race conditions
|
||||
- [selector-resilience.md](/docs/reference/tea/knowledge-base.md#selector-resilience) - Robust selectors
|
||||
```
|
||||
|
||||
## Understanding the Scores
|
||||
|
||||
### What Do Scores Mean?
|
||||
|
||||
| Score Range | Interpretation | Action |
|
||||
|-------------|----------------|--------|
|
||||
| **90-100** | Excellent | Minimal changes needed, production-ready |
|
||||
| **80-89** | Good | Minor improvements recommended |
|
||||
| **70-79** | Acceptable | Address recommendations before release |
|
||||
| **60-69** | Needs Improvement | Fix critical issues, apply recommendations |
|
||||
| **< 60** | Critical | Significant refactoring needed |
|
||||
|
||||
### Scoring Criteria
|
||||
|
||||
**Determinism (35 points):**
|
||||
- Tests produce same result every run
|
||||
- No random failures (flakiness)
|
||||
- No environment-dependent behavior
|
||||
|
||||
**Isolation (25 points):**
|
||||
- Tests don't depend on each other
|
||||
- Can run in any order
|
||||
- Clean up after themselves
|
||||
|
||||
**Assertions (20 points):**
|
||||
- Verify actual behavior
|
||||
- Specific and meaningful
|
||||
- Not abstracted away in helpers
|
||||
|
||||
**Structure (10 points):**
|
||||
- Readable and maintainable
|
||||
- Appropriate size
|
||||
- Clear naming
|
||||
|
||||
**Performance (10 points):**
|
||||
- Fast execution
|
||||
- Efficient selectors
|
||||
- No unnecessary waits
|
||||
|
||||
## What You Get
|
||||
|
||||
### Quality Report
|
||||
- Overall score (0-100)
|
||||
- Category scores (Determinism, Isolation, etc.)
|
||||
- File-by-file breakdown
|
||||
|
||||
### Critical Issues
|
||||
- Specific line numbers
|
||||
- Code examples (current vs fixed)
|
||||
- Why it matters explanation
|
||||
- Impact assessment
|
||||
|
||||
### Recommendations
|
||||
- Actionable improvements
|
||||
- Code examples
|
||||
- Priority/severity levels
|
||||
|
||||
### Next Steps
|
||||
- Immediate actions (fix critical)
|
||||
- Short-term improvements
|
||||
- Long-term quality goals
|
||||
|
||||
## Tips
|
||||
|
||||
### Review Before Release
|
||||
|
||||
Make test review part of release checklist:
|
||||
|
||||
```markdown
|
||||
## Release Checklist
|
||||
- [ ] All tests passing
|
||||
- [ ] Test review score > 80
|
||||
- [ ] Critical issues resolved
|
||||
- [ ] Performance within budget
|
||||
```
|
||||
|
||||
### Review After AI Generation
|
||||
|
||||
Always review AI-generated tests:
|
||||
|
||||
```
|
||||
1. Run *atdd or *automate
|
||||
2. Run *test-review on generated tests
|
||||
3. Fix critical issues
|
||||
4. Commit tests
|
||||
```
|
||||
|
||||
### Set Quality Gates
|
||||
|
||||
Use scores as quality gates:
|
||||
|
||||
```yaml
|
||||
# .github/workflows/test.yml
|
||||
- name: Review test quality
|
||||
run: |
|
||||
# Run test review
|
||||
# Parse score from report
|
||||
if [ $SCORE -lt 80 ]; then
|
||||
echo "Test quality below threshold"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
### Review Regularly
|
||||
|
||||
Schedule periodic reviews:
|
||||
|
||||
- **Per story:** Optional (spot check new tests)
|
||||
- **Per epic:** Recommended (ensure consistency)
|
||||
- **Per release:** Recommended for quality gates (required if using formal gate process)
|
||||
- **Quarterly:** Audit entire suite
|
||||
|
||||
### Focus Reviews
|
||||
|
||||
For large suites, review incrementally:
|
||||
|
||||
**Week 1:** Review E2E tests
|
||||
**Week 2:** Review API tests
|
||||
**Week 3:** Review component tests (Cypress CT or Vitest)
|
||||
**Week 4:** Apply fixes across all suites
|
||||
|
||||
**Component Testing Note:** TEA reviews component tests using framework-specific knowledge:
|
||||
- **Cypress:** Reviews Cypress Component Testing specs (*.cy.tsx)
|
||||
- **Playwright:** Reviews Vitest component tests (*.test.tsx)
|
||||
|
||||
### Use Reviews for Learning
|
||||
|
||||
Share reports with team:
|
||||
|
||||
```
|
||||
Team Meeting:
|
||||
- Review test-review.md
|
||||
- Discuss critical issues
|
||||
- Agree on patterns
|
||||
- Update team guidelines
|
||||
```
|
||||
|
||||
### Compare Over Time
|
||||
|
||||
Track improvement:
|
||||
|
||||
```markdown
|
||||
## Quality Trend
|
||||
|
||||
| Date | Score | Critical Issues | Notes |
|
||||
|------|-------|-----------------|-------|
|
||||
| 2026-01-01 | 65 | 5 | Baseline |
|
||||
| 2026-01-15 | 72 | 2 | Fixed hard waits |
|
||||
| 2026-02-01 | 84 | 0 | All critical resolved |
|
||||
```
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Low Determinism Score
|
||||
|
||||
**Symptoms:**
|
||||
- Tests fail randomly
|
||||
- "Works on my machine"
|
||||
- CI failures that don't reproduce locally
|
||||
|
||||
**Common Causes:**
|
||||
- Hard waits (`waitForTimeout`)
|
||||
- Conditional flow control (`if/else`)
|
||||
- Try-catch for flow control
|
||||
- Missing network-first patterns
|
||||
|
||||
**Fix:** Review determinism section, apply network-first patterns
|
||||
|
||||
### Low Performance Score
|
||||
|
||||
**Symptoms:**
|
||||
- Tests take > 1.5 minutes each
|
||||
- Test suite takes hours
|
||||
- CI times out
|
||||
|
||||
**Common Causes:**
|
||||
- Unnecessary waits (hard timeouts)
|
||||
- Inefficient selectors (XPath, complex CSS)
|
||||
- Not using parallelization
|
||||
- Heavy setup in every test
|
||||
|
||||
**Fix:** Optimize waits, improve selectors, use fixtures
|
||||
|
||||
### Low Isolation Score
|
||||
|
||||
**Symptoms:**
|
||||
- Tests fail when run in different order
|
||||
- Tests fail in parallel
|
||||
- Test data conflicts
|
||||
|
||||
**Common Causes:**
|
||||
- Shared global state
|
||||
- Tests don't clean up
|
||||
- Hard-coded test data
|
||||
- Database not reset between tests
|
||||
|
||||
**Fix:** Use fixtures, clean up in afterEach, use unique test data
|
||||
|
||||
### "Too Many Issues to Fix"
|
||||
|
||||
**Problem:** Report shows 50+ issues, overwhelming.
|
||||
|
||||
**Solution:** Prioritize:
|
||||
1. Fix all critical issues first
|
||||
2. Apply top 3 recommendations
|
||||
3. Re-run review
|
||||
4. Iterate
|
||||
|
||||
Don't try to fix everything at once.
|
||||
|
||||
### Reviews Take Too Long
|
||||
|
||||
**Problem:** Reviewing entire suite takes hours.
|
||||
|
||||
**Solution:** Review incrementally:
|
||||
- Review new tests in PR review
|
||||
- Schedule directory reviews weekly
|
||||
- Full suite review quarterly
|
||||
|
||||
## Related Guides
|
||||
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Generate tests to review
|
||||
- [How to Run Automate](/docs/how-to/workflows/run-automate.md) - Expand coverage to review
|
||||
- [How to Run Trace](/docs/how-to/workflows/run-trace.md) - Coverage complements quality
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - What makes tests good
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Avoiding flakiness
|
||||
- [Fixture Architecture](/docs/explanation/tea/fixture-architecture.md) - Reusable patterns
|
||||
|
||||
## Reference
|
||||
|
||||
- [Command: *test-review](/docs/reference/tea/commands.md#test-review) - Full command reference
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Patterns TEA reviews against
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,883 @@
|
|||
---
|
||||
title: "How to Run Trace with TEA"
|
||||
description: Map requirements to tests and make quality gate decisions using TEA's trace workflow
|
||||
---
|
||||
|
||||
# How to Run Trace with TEA
|
||||
|
||||
Use TEA's `*trace` workflow for requirements traceability and quality gate decisions. This is a two-phase workflow: Phase 1 analyzes coverage, Phase 2 makes the go/no-go decision.
|
||||
|
||||
## When to Use This
|
||||
|
||||
### Phase 1: Requirements Traceability
|
||||
- Map acceptance criteria to implemented tests
|
||||
- Identify coverage gaps
|
||||
- Prioritize missing tests
|
||||
- Refresh coverage after each story/epic
|
||||
|
||||
### Phase 2: Quality Gate Decision
|
||||
- Make go/no-go decision for release
|
||||
- Validate coverage meets thresholds
|
||||
- Document gate decision with evidence
|
||||
- Support business-approved waivers
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed
|
||||
- TEA agent available
|
||||
- Requirements defined (stories, acceptance criteria, test design)
|
||||
- Tests implemented
|
||||
- For brownfield: Existing codebase with tests
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Run the Trace Workflow
|
||||
|
||||
```
|
||||
*trace
|
||||
```
|
||||
|
||||
### 2. Specify Phase
|
||||
|
||||
TEA will ask which phase you're running.
|
||||
|
||||
**Phase 1: Requirements Traceability**
|
||||
- Analyze coverage
|
||||
- Identify gaps
|
||||
- Generate recommendations
|
||||
|
||||
**Phase 2: Quality Gate Decision**
|
||||
- Make PASS/CONCERNS/FAIL/WAIVED decision
|
||||
- Requires Phase 1 complete
|
||||
|
||||
**Typical flow:** Run Phase 1 first, review gaps, then run Phase 2 for gate decision.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Requirements Traceability
|
||||
|
||||
### 3. Provide Requirements Source
|
||||
|
||||
TEA will ask where requirements are defined.
|
||||
|
||||
**Options:**
|
||||
|
||||
| Source | Example | Best For |
|
||||
| --------------- | ----------------------------- | ---------------------- |
|
||||
| **Story file** | `story-profile-management.md` | Single story coverage |
|
||||
| **Test design** | `test-design-epic-1.md` | Epic coverage |
|
||||
| **PRD** | `PRD.md` | System-level coverage |
|
||||
| **Multiple** | All of the above | Comprehensive analysis |
|
||||
|
||||
**Example Response:**
|
||||
```
|
||||
Requirements:
|
||||
- story-profile-management.md (acceptance criteria)
|
||||
- test-design-epic-1.md (test priorities)
|
||||
```
|
||||
|
||||
### 4. Specify Test Location
|
||||
|
||||
TEA will ask where tests are located.
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Test location: tests/
|
||||
Include:
|
||||
- tests/api/
|
||||
- tests/e2e/
|
||||
```
|
||||
|
||||
### 5. Specify Focus Areas (Optional)
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Focus on:
|
||||
- Profile CRUD operations
|
||||
- Validation scenarios
|
||||
- Authorization checks
|
||||
```
|
||||
|
||||
### 6. Review Coverage Matrix
|
||||
|
||||
TEA generates a comprehensive traceability matrix.
|
||||
|
||||
#### Traceability Matrix (`traceability-matrix.md`):
|
||||
|
||||
```markdown
|
||||
# Requirements Traceability Matrix
|
||||
|
||||
**Date:** 2026-01-13
|
||||
**Scope:** Epic 1 - User Profile Management
|
||||
**Phase:** Phase 1 (Traceability Analysis)
|
||||
|
||||
## Coverage Summary
|
||||
|
||||
| Metric | Count | Percentage |
|
||||
| ---------------------- | ----- | ---------- |
|
||||
| **Total Requirements** | 15 | 100% |
|
||||
| **Full Coverage** | 11 | 73% |
|
||||
| **Partial Coverage** | 3 | 20% |
|
||||
| **No Coverage** | 1 | 7% |
|
||||
|
||||
### By Priority
|
||||
|
||||
| Priority | Total | Covered | Percentage |
|
||||
| -------- | ----- | ------- | ----------------- |
|
||||
| **P0** | 5 | 5 | 100% ✅ |
|
||||
| **P1** | 6 | 5 | 83% ⚠️ |
|
||||
| **P2** | 3 | 1 | 33% ⚠️ |
|
||||
| **P3** | 1 | 0 | 0% ✅ (acceptable) |
|
||||
|
||||
---
|
||||
|
||||
## Detailed Traceability
|
||||
|
||||
### ✅ Requirement 1: User can view their profile (P0)
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- User navigates to /profile
|
||||
- Profile displays name, email, avatar
|
||||
- Data is current (not cached)
|
||||
|
||||
**Test Coverage:** FULL ✅
|
||||
|
||||
**Tests:**
|
||||
- `tests/e2e/profile-view.spec.ts:15` - "should display profile page with current data"
|
||||
- ✅ Navigates to /profile
|
||||
- ✅ Verifies name, email visible
|
||||
- ✅ Verifies avatar displayed
|
||||
- ✅ Validates data freshness via API assertion
|
||||
|
||||
- `tests/api/profile.spec.ts:8` - "should fetch user profile via API"
|
||||
- ✅ Calls GET /api/profile
|
||||
- ✅ Validates response schema
|
||||
- ✅ Confirms all fields present
|
||||
|
||||
---
|
||||
|
||||
### ⚠️ Requirement 2: User can edit profile (P0)
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- User clicks "Edit Profile"
|
||||
- Can modify name, email, bio
|
||||
- Can upload avatar
|
||||
- Changes are persisted
|
||||
- Success message shown
|
||||
|
||||
**Test Coverage:** PARTIAL ⚠️
|
||||
|
||||
**Tests:**
|
||||
- `tests/e2e/profile-edit.spec.ts:22` - "should edit and save profile"
|
||||
- ✅ Clicks edit button
|
||||
- ✅ Modifies name and email
|
||||
- ⚠️ **Does NOT test bio field**
|
||||
- ❌ **Does NOT test avatar upload**
|
||||
- ✅ Verifies persistence
|
||||
- ✅ Verifies success message
|
||||
|
||||
- `tests/api/profile.spec.ts:25` - "should update profile via PATCH"
|
||||
- ✅ Calls PATCH /api/profile
|
||||
- ✅ Validates update response
|
||||
- ⚠️ **Only tests name/email, not bio/avatar**
|
||||
|
||||
**Missing Coverage:**
|
||||
- Bio field not tested in E2E or API
|
||||
- Avatar upload not tested
|
||||
|
||||
**Gap Severity:** HIGH (P0 requirement, critical path)
|
||||
|
||||
---
|
||||
|
||||
### ✅ Requirement 3: Invalid email shows validation error (P1)
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- Enter invalid email format
|
||||
- See error message
|
||||
- Cannot save changes
|
||||
|
||||
**Test Coverage:** FULL ✅
|
||||
|
||||
**Tests:**
|
||||
- `tests/e2e/profile-edit.spec.ts:45` - "should show validation error for invalid email"
|
||||
- `tests/api/profile.spec.ts:50` - "should return 400 for invalid email"
|
||||
|
||||
---
|
||||
|
||||
### ❌ Requirement 15: Profile export as PDF (P2)
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- User clicks "Export Profile"
|
||||
- PDF downloads with profile data
|
||||
|
||||
**Test Coverage:** NONE ❌
|
||||
|
||||
**Gap Analysis:**
|
||||
- **Priority:** P2 (medium)
|
||||
- **Risk:** Low (non-critical feature)
|
||||
- **Recommendation:** Add in next iteration (not blocking for release)
|
||||
|
||||
---
|
||||
|
||||
## Gap Prioritization
|
||||
|
||||
### Critical Gaps (Must Fix Before Release)
|
||||
|
||||
| Gap | Requirement | Priority | Risk | Recommendation |
|
||||
| --- | ------------------------ | -------- | ---- | ------------------- |
|
||||
| 1 | Bio field not tested | P0 | High | Add E2E + API tests |
|
||||
| 2 | Avatar upload not tested | P0 | High | Add E2E + API tests |
|
||||
|
||||
**Estimated Effort:** 3 hours
|
||||
**Owner:** QA team
|
||||
**Deadline:** Before release
|
||||
|
||||
### Non-Critical Gaps (Can Defer)
|
||||
|
||||
| Gap | Requirement | Priority | Risk | Recommendation |
|
||||
| --- | ------------------------- | -------- | ---- | ------------------- |
|
||||
| 3 | Profile export not tested | P2 | Low | Add in v1.3 release |
|
||||
|
||||
**Estimated Effort:** 2 hours
|
||||
**Owner:** QA team
|
||||
**Deadline:** Next release (February)
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
### 1. Add Bio Field Tests
|
||||
|
||||
**Tests Needed (Vanilla Playwright):**
|
||||
```typescript
|
||||
// tests/e2e/profile-edit.spec.ts
|
||||
test('should edit bio field', async ({ page }) => {
|
||||
await page.goto('/profile');
|
||||
await page.getByRole('button', { name: 'Edit' }).click();
|
||||
await page.getByLabel('Bio').fill('New bio text');
|
||||
await page.getByRole('button', { name: 'Save' }).click();
|
||||
await expect(page.getByText('New bio text')).toBeVisible();
|
||||
});
|
||||
|
||||
// tests/api/profile.spec.ts
|
||||
test('should update bio via API', async ({ request }) => {
|
||||
const response = await request.patch('/api/profile', {
|
||||
data: { bio: 'Updated bio' }
|
||||
});
|
||||
expect(response.ok()).toBeTruthy();
|
||||
const { bio } = await response.json();
|
||||
expect(bio).toBe('Updated bio');
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
// tests/e2e/profile-edit.spec.ts
|
||||
import { test } from '../support/fixtures'; // Composed with authToken
|
||||
|
||||
test('should edit bio field', async ({ page, authToken }) => {
|
||||
await page.goto('/profile');
|
||||
await page.getByRole('button', { name: 'Edit' }).click();
|
||||
await page.getByLabel('Bio').fill('New bio text');
|
||||
await page.getByRole('button', { name: 'Save' }).click();
|
||||
await expect(page.getByText('New bio text')).toBeVisible();
|
||||
});
|
||||
|
||||
// tests/api/profile.spec.ts
|
||||
import { test as base, expect } from '@playwright/test';
|
||||
import { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { createAuthFixtures } from '@seontechnologies/playwright-utils/auth-session';
|
||||
import { mergeTests } from '@playwright/test';
|
||||
|
||||
// Merge API request + auth fixtures
|
||||
const authFixtureTest = base.extend(createAuthFixtures());
|
||||
const test = mergeTests(apiRequestFixture, authFixtureTest);
|
||||
|
||||
test('should update bio via API', async ({ apiRequest, authToken }) => {
|
||||
const { status, body } = await apiRequest({
|
||||
method: 'PATCH',
|
||||
path: '/api/profile',
|
||||
body: { bio: 'Updated bio' },
|
||||
headers: { Authorization: `Bearer ${authToken}` }
|
||||
});
|
||||
|
||||
expect(status).toBe(200);
|
||||
expect(body.bio).toBe('Updated bio');
|
||||
});
|
||||
```
|
||||
|
||||
**Note:** `authToken` requires auth-session fixture setup. See [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md#auth-session).
|
||||
|
||||
### 2. Add Avatar Upload Tests
|
||||
|
||||
**Tests Needed:**
|
||||
```typescript
|
||||
// tests/e2e/profile-edit.spec.ts
|
||||
test('should upload avatar image', async ({ page }) => {
|
||||
await page.goto('/profile');
|
||||
await page.getByRole('button', { name: 'Edit' }).click();
|
||||
|
||||
// Upload file
|
||||
await page.setInputFiles('[type="file"]', 'fixtures/avatar.png');
|
||||
await page.getByRole('button', { name: 'Save' }).click();
|
||||
|
||||
// Verify uploaded image displays
|
||||
await expect(page.locator('img[alt="Profile avatar"]')).toBeVisible();
|
||||
});
|
||||
|
||||
// tests/api/profile.spec.ts
|
||||
import { test, expect } from '@playwright/test';
|
||||
import fs from 'fs/promises';
|
||||
|
||||
test('should accept valid image upload', async ({ request }) => {
|
||||
const response = await request.post('/api/profile/avatar', {
|
||||
multipart: {
|
||||
file: {
|
||||
name: 'avatar.png',
|
||||
mimeType: 'image/png',
|
||||
buffer: await fs.readFile('fixtures/avatar.png')
|
||||
}
|
||||
}
|
||||
});
|
||||
expect(response.ok()).toBeTruthy();
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
After reviewing traceability:
|
||||
|
||||
1. **Fix critical gaps** - Add tests for P0/P1 requirements
|
||||
2. **Run *test-review** - Ensure new tests meet quality standards
|
||||
3. **Run Phase 2** - Make gate decision after gaps addressed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Quality Gate Decision
|
||||
|
||||
After Phase 1 coverage analysis is complete, run Phase 2 for the gate decision.
|
||||
|
||||
**Prerequisites:**
|
||||
- Phase 1 traceability matrix complete
|
||||
- Test execution results available (must have test results)
|
||||
|
||||
**Note:** Phase 2 will skip if test execution results aren't provided. The workflow requires actual test run results to make gate decisions.
|
||||
|
||||
### 7. Run Phase 2
|
||||
|
||||
```
|
||||
*trace
|
||||
```
|
||||
|
||||
Select "Phase 2: Quality Gate Decision"
|
||||
|
||||
### 8. Provide Additional Context
|
||||
|
||||
TEA will ask for:
|
||||
|
||||
**Gate Type:**
|
||||
- Story gate (small release)
|
||||
- Epic gate (larger release)
|
||||
- Release gate (production deployment)
|
||||
- Hotfix gate (emergency fix)
|
||||
|
||||
**Decision Mode:**
|
||||
- **Deterministic** - Rule-based (coverage %, quality scores)
|
||||
- **Manual** - Team decision with TEA guidance
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Gate type: Epic gate
|
||||
Decision mode: Deterministic
|
||||
```
|
||||
|
||||
### 9. Provide Supporting Evidence
|
||||
|
||||
TEA will request:
|
||||
|
||||
**Phase 1 Results:**
|
||||
```
|
||||
traceability-matrix.md (from Phase 1)
|
||||
```
|
||||
|
||||
**Test Quality (Optional):**
|
||||
```
|
||||
test-review.md (from *test-review)
|
||||
```
|
||||
|
||||
**NFR Assessment (Optional):**
|
||||
```
|
||||
nfr-assessment.md (from *nfr-assess)
|
||||
```
|
||||
|
||||
### 10. Review Gate Decision
|
||||
|
||||
TEA makes evidence-based gate decision and writes to separate file.
|
||||
|
||||
#### Gate Decision (`gate-decision-{gate_type}-{story_id}.md`):
|
||||
|
||||
```markdown
|
||||
---
|
||||
|
||||
# Phase 2: Quality Gate Decision
|
||||
|
||||
**Gate Type:** Epic Gate
|
||||
**Decision:** PASS ✅
|
||||
**Date:** 2026-01-13
|
||||
**Approvers:** Product Manager, Tech Lead, QA Lead
|
||||
|
||||
## Decision Summary
|
||||
|
||||
**Verdict:** Ready to release
|
||||
|
||||
**Evidence:**
|
||||
- P0 coverage: 100% (5/5 requirements)
|
||||
- P1 coverage: 100% (6/6 requirements)
|
||||
- P2 coverage: 33% (1/3 requirements) - acceptable
|
||||
- Test quality score: 84/100
|
||||
- NFR assessment: PASS
|
||||
|
||||
## Coverage Analysis
|
||||
|
||||
| Priority | Required Coverage | Actual Coverage | Status |
|
||||
| -------- | ----------------- | --------------- | --------------------- |
|
||||
| **P0** | 100% | 100% | ✅ PASS |
|
||||
| **P1** | 90% | 100% | ✅ PASS |
|
||||
| **P2** | 50% | 33% | ⚠️ Below (acceptable) |
|
||||
| **P3** | 20% | 0% | ✅ PASS (low priority) |
|
||||
|
||||
**Rationale:**
|
||||
- All critical path (P0) requirements fully tested
|
||||
- All high-value (P1) requirements fully tested
|
||||
- P2 gap (profile export) is low risk and deferred to next release
|
||||
|
||||
## Quality Metrics
|
||||
|
||||
| Metric | Threshold | Actual | Status |
|
||||
| ------------------ | --------- | ------ | ------ |
|
||||
| P0/P1 Coverage | >95% | 100% | ✅ |
|
||||
| Test Quality Score | >80 | 84 | ✅ |
|
||||
| NFR Status | PASS | PASS | ✅ |
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
### Accepted Risks
|
||||
|
||||
**Risk 1: Profile export not tested (P2)**
|
||||
- **Impact:** Medium (users can't export profile)
|
||||
- **Mitigation:** Feature flag disabled by default
|
||||
- **Plan:** Add tests in v1.3 release (February)
|
||||
- **Monitoring:** Track feature flag usage
|
||||
|
||||
## Approvals
|
||||
|
||||
- [x] **Product Manager** - Business requirements met (Approved: 2026-01-13)
|
||||
- [x] **Tech Lead** - Technical quality acceptable (Approved: 2026-01-13)
|
||||
- [x] **QA Lead** - Test coverage sufficient (Approved: 2026-01-13)
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Deployment
|
||||
1. Merge to main branch
|
||||
2. Deploy to staging
|
||||
3. Run smoke tests in staging
|
||||
4. Deploy to production
|
||||
5. Monitor for 24 hours
|
||||
|
||||
### Monitoring
|
||||
- Set alerts for profile endpoint (P99 > 200ms)
|
||||
- Track error rates (target: <0.1%)
|
||||
- Monitor profile export feature flag usage
|
||||
|
||||
### Future Work
|
||||
- Add profile export tests (v1.3)
|
||||
- Expand P2 coverage to 50%
|
||||
```
|
||||
|
||||
### Gate Decision Rules
|
||||
|
||||
TEA uses deterministic rules when decision_mode = "deterministic":
|
||||
|
||||
| P0 Coverage | P1 Coverage | Overall Coverage | Decision |
|
||||
| ----------- | ----------- | ---------------- | ---------------------------- |
|
||||
| 100% | ≥90% | ≥80% | **PASS** ✅ |
|
||||
| 100% | 80-89% | ≥80% | **CONCERNS** ⚠️ |
|
||||
| <100% | Any | Any | **FAIL** ❌ |
|
||||
| Any | <80% | Any | **FAIL** ❌ |
|
||||
| Any | Any | <80% | **FAIL** ❌ |
|
||||
| Any | Any | Any | **WAIVED** ⏭️ (with approval) |
|
||||
|
||||
**Detailed Rules:**
|
||||
- **PASS:** P0=100%, P1≥90%, Overall≥80%
|
||||
- **CONCERNS:** P0=100%, P1 80-89%, Overall≥80% (below threshold but not critical)
|
||||
- **FAIL:** P0<100% OR P1<80% OR Overall<80% (critical gaps)
|
||||
|
||||
**PASS** ✅: All criteria met, ready to release
|
||||
|
||||
**CONCERNS** ⚠️: Some criteria not met, but:
|
||||
- Mitigation plan exists
|
||||
- Risk is acceptable
|
||||
- Team approves proceeding
|
||||
- Monitoring in place
|
||||
|
||||
**FAIL** ❌: Critical criteria not met:
|
||||
- P0 requirements not tested
|
||||
- Critical security vulnerabilities
|
||||
- System is broken
|
||||
- Cannot deploy
|
||||
|
||||
**WAIVED** ⏭️: Business approves proceeding despite concerns:
|
||||
- Documented business justification
|
||||
- Accepted risks quantified
|
||||
- Approver signatures
|
||||
- Future plans documented
|
||||
|
||||
### Example CONCERNS Decision
|
||||
|
||||
```markdown
|
||||
## Decision Summary
|
||||
|
||||
**Verdict:** CONCERNS ⚠️ - Proceed with monitoring
|
||||
|
||||
**Evidence:**
|
||||
- P0 coverage: 100%
|
||||
- P1 coverage: 85% (below 90% target)
|
||||
- Test quality: 78/100 (below 80 target)
|
||||
|
||||
**Gaps:**
|
||||
- 1 P1 requirement not tested (avatar upload)
|
||||
- Test quality score slightly below threshold
|
||||
|
||||
**Mitigation:**
|
||||
- Avatar upload not critical for v1.2 launch
|
||||
- Test quality issues are minor (no flakiness)
|
||||
- Monitoring alerts configured
|
||||
|
||||
**Approvals:**
|
||||
- Product Manager: APPROVED (business priority to launch)
|
||||
- Tech Lead: APPROVED (technical risk acceptable)
|
||||
```
|
||||
|
||||
### Example FAIL Decision
|
||||
|
||||
```markdown
|
||||
## Decision Summary
|
||||
|
||||
**Verdict:** FAIL ❌ - Cannot release
|
||||
|
||||
**Evidence:**
|
||||
- P0 coverage: 60% (below 95% threshold)
|
||||
- Critical security vulnerability (CVE-2024-12345)
|
||||
- Test quality: 55/100
|
||||
|
||||
**Blockers:**
|
||||
1. **Login flow not tested** (P0 requirement)
|
||||
- Critical path completely untested
|
||||
- Must add E2E and API tests
|
||||
|
||||
2. **SQL injection vulnerability**
|
||||
- Critical security issue
|
||||
- Must fix before deployment
|
||||
|
||||
**Actions Required:**
|
||||
1. Add login tests (QA team, 2 days)
|
||||
2. Fix SQL injection (backend team, 1 day)
|
||||
3. Re-run security scan (DevOps, 1 hour)
|
||||
4. Re-run *trace after fixes
|
||||
|
||||
**Cannot proceed until all blockers resolved.**
|
||||
```
|
||||
|
||||
## What You Get
|
||||
|
||||
### Phase 1: Traceability Matrix
|
||||
- Requirement-to-test mapping
|
||||
- Coverage classification (FULL/PARTIAL/NONE)
|
||||
- Gap identification with priorities
|
||||
- Actionable recommendations
|
||||
|
||||
### Phase 2: Gate Decision
|
||||
- Go/no-go verdict (PASS/CONCERNS/FAIL/WAIVED)
|
||||
- Evidence summary
|
||||
- Approval signatures
|
||||
- Next steps and monitoring plan
|
||||
|
||||
## Usage Patterns
|
||||
|
||||
### Greenfield Projects
|
||||
|
||||
**Phase 3:**
|
||||
```
|
||||
After architecture complete:
|
||||
1. Run *test-design (system-level)
|
||||
2. Run *trace Phase 1 (baseline)
|
||||
3. Use for implementation-readiness gate
|
||||
```
|
||||
|
||||
**Phase 4:**
|
||||
```
|
||||
After each epic/story:
|
||||
1. Run *trace Phase 1 (refresh coverage)
|
||||
2. Identify gaps
|
||||
3. Add missing tests
|
||||
```
|
||||
|
||||
**Release Gate:**
|
||||
```
|
||||
Before deployment:
|
||||
1. Run *trace Phase 1 (final coverage check)
|
||||
2. Run *trace Phase 2 (make gate decision)
|
||||
3. Get approvals
|
||||
4. Deploy (if PASS or WAIVED)
|
||||
```
|
||||
|
||||
### Brownfield Projects
|
||||
|
||||
**Phase 2:**
|
||||
```
|
||||
Before planning new work:
|
||||
1. Run *trace Phase 1 (establish baseline)
|
||||
2. Understand existing coverage
|
||||
3. Plan testing strategy
|
||||
```
|
||||
|
||||
**Phase 4:**
|
||||
```
|
||||
After each epic/story:
|
||||
1. Run *trace Phase 1 (refresh)
|
||||
2. Compare to baseline
|
||||
3. Track coverage improvement
|
||||
```
|
||||
|
||||
**Release Gate:**
|
||||
```
|
||||
Before deployment:
|
||||
1. Run *trace Phase 1 (final check)
|
||||
2. Run *trace Phase 2 (gate decision)
|
||||
3. Compare to baseline
|
||||
4. Deploy if coverage maintained or improved
|
||||
```
|
||||
|
||||
## Tips
|
||||
|
||||
### Run Phase 1 Frequently
|
||||
|
||||
Don't wait until release gate:
|
||||
|
||||
```
|
||||
After Story 1: *trace Phase 1 (identify gaps early)
|
||||
After Story 2: *trace Phase 1 (refresh)
|
||||
After Story 3: *trace Phase 1 (refresh)
|
||||
Before Release: *trace Phase 1 + Phase 2 (final gate)
|
||||
```
|
||||
|
||||
**Benefit:** Catch gaps early when they're cheap to fix.
|
||||
|
||||
### Use Coverage Trends
|
||||
|
||||
Track improvement over time:
|
||||
|
||||
```markdown
|
||||
## Coverage Trend
|
||||
|
||||
| Date | Epic | P0/P1 Coverage | Quality Score | Status |
|
||||
| ---------- | -------- | -------------- | ------------- | -------------- |
|
||||
| 2026-01-01 | Baseline | 45% | - | Starting point |
|
||||
| 2026-01-08 | Epic 1 | 78% | 72 | Improving |
|
||||
| 2026-01-15 | Epic 2 | 92% | 84 | Near target |
|
||||
| 2026-01-20 | Epic 3 | 100% | 88 | Ready! |
|
||||
```
|
||||
|
||||
### Set Coverage Targets by Priority
|
||||
|
||||
Don't aim for 100% across all priorities:
|
||||
|
||||
**Recommended Targets:**
|
||||
- **P0:** 100% (critical path must be tested)
|
||||
- **P1:** 90% (high-value scenarios)
|
||||
- **P2:** 50% (nice-to-have features)
|
||||
- **P3:** 20% (low-value edge cases)
|
||||
|
||||
### Use Classification Strategically
|
||||
|
||||
**FULL** ✅: Requirement completely tested
|
||||
- E2E test covers full user workflow
|
||||
- API test validates backend behavior
|
||||
- All acceptance criteria covered
|
||||
|
||||
**PARTIAL** ⚠️: Some aspects tested
|
||||
- E2E test exists but missing scenarios
|
||||
- API test exists but incomplete
|
||||
- Some acceptance criteria not covered
|
||||
|
||||
**NONE** ❌: No tests exist
|
||||
- Requirement identified but not tested
|
||||
- May be intentional (low priority) or oversight
|
||||
|
||||
**Classification helps prioritize:**
|
||||
- Fix NONE coverage for P0/P1 requirements first
|
||||
- Enhance PARTIAL coverage for P0 requirements
|
||||
- Accept PARTIAL or NONE for P2/P3 if time-constrained
|
||||
|
||||
### Automate Gate Decisions
|
||||
|
||||
Use traceability in CI:
|
||||
|
||||
```yaml
|
||||
# .github/workflows/gate-check.yml
|
||||
- name: Check coverage
|
||||
run: |
|
||||
# Run trace Phase 1
|
||||
# Parse coverage percentages
|
||||
if [ $P0_COVERAGE -lt 95 ]; then
|
||||
echo "P0 coverage below 95%"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
### Document Waivers Clearly
|
||||
|
||||
If proceeding with WAIVED:
|
||||
|
||||
**Required:**
|
||||
```markdown
|
||||
## Waiver Documentation
|
||||
|
||||
**Waived By:** VP Engineering, Product Lead
|
||||
**Date:** 2026-01-15
|
||||
**Gate Type:** Release Gate v1.2
|
||||
|
||||
**Justification:**
|
||||
Business critical to launch by Q1 for investor demo.
|
||||
Performance concerns acceptable for initial user base.
|
||||
|
||||
**Conditions:**
|
||||
- Set monitoring alerts for P99 > 300ms
|
||||
- Plan optimization for v1.3 (due February 28)
|
||||
- Monitor user feedback closely
|
||||
|
||||
**Accepted Risks:**
|
||||
- 1% of users may experience 350ms latency
|
||||
- Avatar upload feature incomplete
|
||||
- Profile export deferred to next release
|
||||
|
||||
**Quantified Impact:**
|
||||
- Affects <100 users at current scale
|
||||
- Workaround exists (manual export)
|
||||
- Monitoring will catch issues early
|
||||
|
||||
**Approvals:**
|
||||
- VP Engineering: [Signature] Date: 2026-01-15
|
||||
- Product Lead: [Signature] Date: 2026-01-15
|
||||
- QA Lead: [Signature] Date: 2026-01-15
|
||||
```
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Too Many Gaps to Fix
|
||||
|
||||
**Problem:** Phase 1 shows 50 uncovered requirements.
|
||||
|
||||
**Solution:** Prioritize ruthlessly:
|
||||
1. Fix all P0 gaps (critical path)
|
||||
2. Fix high-risk P1 gaps
|
||||
3. Accept low-risk P1 gaps with mitigation
|
||||
4. Defer all P2/P3 gaps
|
||||
|
||||
**Don't try to fix everything** - focus on what matters for release.
|
||||
|
||||
### Can't Find Test Coverage
|
||||
|
||||
**Problem:** Tests exist but TEA can't map them to requirements.
|
||||
|
||||
**Cause:** Tests don't reference requirements.
|
||||
|
||||
**Solution:** Add traceability comments:
|
||||
```typescript
|
||||
test('should display profile', async ({ page }) => {
|
||||
// Covers: Requirement 1 - User can view profile
|
||||
// Acceptance criteria: Navigate to /profile, see name/email
|
||||
await page.goto('/profile');
|
||||
await expect(page.getByText('Test User')).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
Or use test IDs:
|
||||
```typescript
|
||||
test('[REQ-1] should display profile', async ({ page }) => {
|
||||
// Test code...
|
||||
});
|
||||
```
|
||||
|
||||
### Unclear What "FULL" vs "PARTIAL" Means
|
||||
|
||||
**FULL** ✅: All acceptance criteria tested
|
||||
```
|
||||
Requirement: User can edit profile
|
||||
Acceptance criteria:
|
||||
- Can modify name ✅ Tested
|
||||
- Can modify email ✅ Tested
|
||||
- Can upload avatar ✅ Tested
|
||||
- Changes persist ✅ Tested
|
||||
Result: FULL coverage
|
||||
```
|
||||
|
||||
**PARTIAL** ⚠️: Some criteria tested, some not
|
||||
```
|
||||
Requirement: User can edit profile
|
||||
Acceptance criteria:
|
||||
- Can modify name ✅ Tested
|
||||
- Can modify email ✅ Tested
|
||||
- Can upload avatar ❌ Not tested
|
||||
- Changes persist ✅ Tested
|
||||
Result: PARTIAL coverage (3/4 criteria)
|
||||
```
|
||||
|
||||
### Gate Decision Unclear
|
||||
|
||||
**Problem:** Not sure if PASS or CONCERNS is appropriate.
|
||||
|
||||
**Guideline:**
|
||||
|
||||
**Use PASS** ✅ if:
|
||||
- All P0 requirements 100% covered
|
||||
- P1 requirements >90% covered
|
||||
- No critical issues
|
||||
- NFRs met
|
||||
|
||||
**Use CONCERNS** ⚠️ if:
|
||||
- P1 coverage 85-90% (close to threshold)
|
||||
- Minor quality issues (score 70-79)
|
||||
- NFRs have mitigation plans
|
||||
- Team agrees risk is acceptable
|
||||
|
||||
**Use FAIL** ❌ if:
|
||||
- P0 coverage <100% (critical path gaps)
|
||||
- P1 coverage <85%
|
||||
- Critical security/performance issues
|
||||
- No mitigation possible
|
||||
|
||||
**When in doubt, use CONCERNS** and document the risk.
|
||||
|
||||
## Related Guides
|
||||
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md) - Provides requirements for traceability
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md) - Quality scores feed gate
|
||||
- [How to Run NFR Assessment](/docs/how-to/workflows/run-nfr-assess.md) - NFR status feeds gate
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - Why P0 vs P3 matters
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Gate decisions in context
|
||||
|
||||
## Reference
|
||||
|
||||
- [Command: *trace](/docs/reference/tea/commands.md#trace) - Full command reference
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Config options
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,712 @@
|
|||
---
|
||||
title: "How to Set Up CI Pipeline with TEA"
|
||||
description: Configure automated test execution with selective testing and burn-in loops using TEA
|
||||
---
|
||||
|
||||
# How to Set Up CI Pipeline with TEA
|
||||
|
||||
Use TEA's `*ci` workflow to scaffold production-ready CI/CD configuration for automated test execution with selective testing, parallel sharding, and flakiness detection.
|
||||
|
||||
## When to Use This
|
||||
|
||||
- Need to automate test execution in CI/CD
|
||||
- Want selective testing (only run affected tests)
|
||||
- Need parallel execution for faster feedback
|
||||
- Want burn-in loops for flakiness detection
|
||||
- Setting up new CI/CD pipeline
|
||||
- Optimizing existing CI/CD workflow
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- BMad Method installed
|
||||
- TEA agent available
|
||||
- Test framework configured (run `*framework` first)
|
||||
- Tests written (have something to run in CI)
|
||||
- CI/CD platform access (GitHub Actions, GitLab CI, etc.)
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Load TEA Agent
|
||||
|
||||
Start a fresh chat and load TEA:
|
||||
|
||||
```
|
||||
*tea
|
||||
```
|
||||
|
||||
### 2. Run the CI Workflow
|
||||
|
||||
```
|
||||
*ci
|
||||
```
|
||||
|
||||
### 3. Select CI/CD Platform
|
||||
|
||||
TEA will ask which platform you're using.
|
||||
|
||||
**Supported Platforms:**
|
||||
- **GitHub Actions** (most common)
|
||||
- **GitLab CI**
|
||||
- **Circle CI**
|
||||
- **Jenkins**
|
||||
- **Other** (TEA provides generic template)
|
||||
|
||||
**Example:**
|
||||
```
|
||||
GitHub Actions
|
||||
```
|
||||
|
||||
### 4. Configure Test Strategy
|
||||
|
||||
TEA will ask about your test execution strategy.
|
||||
|
||||
#### Repository Structure
|
||||
|
||||
**Question:** "What's your repository structure?"
|
||||
|
||||
**Options:**
|
||||
- **Single app** - One application in root
|
||||
- **Monorepo** - Multiple apps/packages
|
||||
- **Monorepo with affected detection** - Only test changed packages
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Monorepo with multiple apps
|
||||
Need selective testing for changed packages only
|
||||
```
|
||||
|
||||
#### Parallel Execution
|
||||
|
||||
**Question:** "Want to shard tests for parallel execution?"
|
||||
|
||||
**Options:**
|
||||
- **No sharding** - Run tests sequentially
|
||||
- **Shard by workers** - Split across N workers
|
||||
- **Shard by file** - Each file runs in parallel
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Yes, shard across 4 workers for faster execution
|
||||
```
|
||||
|
||||
**Why Shard?**
|
||||
- **4 workers:** 20-minute suite → 5 minutes
|
||||
- **Better resource usage:** Utilize CI runners efficiently
|
||||
- **Faster feedback:** Developers wait less
|
||||
|
||||
#### Burn-In Loops
|
||||
|
||||
**Question:** "Want burn-in loops for flakiness detection?"
|
||||
|
||||
**Options:**
|
||||
- **No burn-in** - Run tests once
|
||||
- **PR burn-in** - Run tests multiple times on PRs
|
||||
- **Nightly burn-in** - Dedicated flakiness detection job
|
||||
|
||||
**Example:**
|
||||
```
|
||||
Yes, run tests 5 times on PRs to catch flaky tests early
|
||||
```
|
||||
|
||||
**Why Burn-In?**
|
||||
- Catches flaky tests before they merge
|
||||
- Prevents intermittent CI failures
|
||||
- Builds confidence in test suite
|
||||
|
||||
### 5. Review Generated CI Configuration
|
||||
|
||||
TEA generates platform-specific workflow files.
|
||||
|
||||
#### GitHub Actions (`.github/workflows/test.yml`):
|
||||
|
||||
```yaml
|
||||
name: Test Suite
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
push:
|
||||
branches: [main, develop]
|
||||
schedule:
|
||||
- cron: '0 2 * * *' # Nightly at 2 AM
|
||||
|
||||
jobs:
|
||||
# Main test job with sharding
|
||||
test:
|
||||
name: Test (Shard ${{ matrix.shard }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
shard: [1, 2, 3, 4]
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: '.nvmrc'
|
||||
cache: 'npm'
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Install Playwright browsers
|
||||
run: npx playwright install --with-deps
|
||||
|
||||
- name: Run tests
|
||||
run: npx playwright test --shard=${{ matrix.shard }}/4
|
||||
|
||||
- name: Upload test results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: test-results-${{ matrix.shard }}
|
||||
path: test-results/
|
||||
retention-days: 7
|
||||
|
||||
- name: Upload test report
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: playwright-report-${{ matrix.shard }}
|
||||
path: playwright-report/
|
||||
retention-days: 7
|
||||
|
||||
# Burn-in job for flakiness detection (PRs only)
|
||||
burn-in:
|
||||
name: Burn-In (Flakiness Detection)
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request'
|
||||
timeout-minutes: 30
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: '.nvmrc'
|
||||
cache: 'npm'
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Install Playwright browsers
|
||||
run: npx playwright install --with-deps
|
||||
|
||||
- name: Run burn-in loop
|
||||
run: |
|
||||
for i in {1..5}; do
|
||||
echo "=== Burn-in iteration $i/5 ==="
|
||||
npx playwright test --grep-invert "@skip" || exit 1
|
||||
done
|
||||
|
||||
- name: Upload burn-in results
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: burn-in-failures
|
||||
path: test-results/
|
||||
|
||||
# Selective testing (changed files only)
|
||||
selective:
|
||||
name: Selective Tests
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request'
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0 # Full history for git diff
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: '.nvmrc'
|
||||
cache: 'npm'
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Install Playwright browsers
|
||||
run: npx playwright install --with-deps
|
||||
|
||||
- name: Run selective tests
|
||||
run: npm run test:changed
|
||||
```
|
||||
|
||||
#### GitLab CI (`.gitlab-ci.yml`):
|
||||
|
||||
```yaml
|
||||
variables:
|
||||
NODE_VERSION: "18"
|
||||
|
||||
stages:
|
||||
- test
|
||||
- burn-in
|
||||
|
||||
# Test job with parallel execution
|
||||
test:
|
||||
stage: test
|
||||
image: node:$NODE_VERSION
|
||||
parallel: 4
|
||||
script:
|
||||
- npm ci
|
||||
- npx playwright install --with-deps
|
||||
- npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
|
||||
artifacts:
|
||||
when: always
|
||||
paths:
|
||||
- test-results/
|
||||
- playwright-report/
|
||||
expire_in: 7 days
|
||||
rules:
|
||||
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
|
||||
# Burn-in job for flakiness detection
|
||||
burn-in:
|
||||
stage: burn-in
|
||||
image: node:$NODE_VERSION
|
||||
script:
|
||||
- npm ci
|
||||
- npx playwright install --with-deps
|
||||
- |
|
||||
for i in {1..5}; do
|
||||
echo "=== Burn-in iteration $i/5 ==="
|
||||
npx playwright test || exit 1
|
||||
done
|
||||
artifacts:
|
||||
when: on_failure
|
||||
paths:
|
||||
- test-results/
|
||||
rules:
|
||||
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
||||
```
|
||||
|
||||
#### Burn-In Testing
|
||||
|
||||
**Option 1: Classic Burn-In (Playwright Built-In)**
|
||||
|
||||
```json
|
||||
{
|
||||
"scripts": {
|
||||
"test": "playwright test",
|
||||
"test:burn-in": "playwright test --repeat-each=5 --retries=0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**How it works:**
|
||||
- Runs every test 5 times
|
||||
- Fails if any iteration fails
|
||||
- Detects flakiness before merge
|
||||
|
||||
**Use when:** Small test suite, want to run everything multiple times
|
||||
|
||||
---
|
||||
|
||||
**Option 2: Smart Burn-In (Playwright Utils)**
|
||||
|
||||
If `tea_use_playwright_utils: true`:
|
||||
|
||||
**scripts/burn-in-changed.ts:**
|
||||
```typescript
|
||||
import { runBurnIn } from '@seontechnologies/playwright-utils/burn-in';
|
||||
|
||||
await runBurnIn({
|
||||
configPath: 'playwright.burn-in.config.ts',
|
||||
baseBranch: 'main'
|
||||
});
|
||||
```
|
||||
|
||||
**playwright.burn-in.config.ts:**
|
||||
```typescript
|
||||
import type { BurnInConfig } from '@seontechnologies/playwright-utils/burn-in';
|
||||
|
||||
const config: BurnInConfig = {
|
||||
skipBurnInPatterns: ['**/config/**', '**/*.md', '**/*types*'],
|
||||
burnInTestPercentage: 0.3,
|
||||
burnIn: { repeatEach: 5, retries: 0 }
|
||||
};
|
||||
|
||||
export default config;
|
||||
```
|
||||
|
||||
**package.json:**
|
||||
```json
|
||||
{
|
||||
"scripts": {
|
||||
"test:burn-in": "tsx scripts/burn-in-changed.ts"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**How it works:**
|
||||
- Git diff analysis (only affected tests)
|
||||
- Smart filtering (skip configs, docs, types)
|
||||
- Volume control (run 30% of affected tests)
|
||||
- Each test runs 5 times
|
||||
|
||||
**Use when:** Large test suite, want intelligent selection
|
||||
|
||||
---
|
||||
|
||||
**Comparison:**
|
||||
|
||||
| Feature | Classic Burn-In | Smart Burn-In (PW-Utils) |
|
||||
|---------|----------------|--------------------------|
|
||||
| Changed 1 file | Runs all 500 tests × 5 = 2500 runs | Runs 3 affected tests × 5 = 15 runs |
|
||||
| Config change | Runs all tests | Skips (no tests affected) |
|
||||
| Type change | Runs all tests | Skips (no runtime impact) |
|
||||
| Setup | Zero config | Requires config file |
|
||||
|
||||
**Recommendation:** Start with classic (simple), upgrade to smart (faster) when suite grows.
|
||||
|
||||
### 6. Configure Secrets
|
||||
|
||||
TEA provides a secrets checklist.
|
||||
|
||||
**Required Secrets** (add to CI/CD platform):
|
||||
|
||||
```markdown
|
||||
## GitHub Actions Secrets
|
||||
|
||||
Repository Settings → Secrets and variables → Actions
|
||||
|
||||
### Required
|
||||
- None (tests run without external auth)
|
||||
|
||||
### Optional
|
||||
- `TEST_USER_EMAIL` - Test user credentials
|
||||
- `TEST_USER_PASSWORD` - Test user password
|
||||
- `API_BASE_URL` - API endpoint for tests
|
||||
- `DATABASE_URL` - Test database (if needed)
|
||||
```
|
||||
|
||||
**How to Add Secrets:**
|
||||
|
||||
**GitHub Actions:**
|
||||
1. Go to repo Settings → Secrets → Actions
|
||||
2. Click "New repository secret"
|
||||
3. Add name and value
|
||||
4. Use in workflow: `${{ secrets.TEST_USER_EMAIL }}`
|
||||
|
||||
**GitLab CI:**
|
||||
1. Go to Project Settings → CI/CD → Variables
|
||||
2. Add variable name and value
|
||||
3. Use in workflow: `$TEST_USER_EMAIL`
|
||||
|
||||
### 7. Test the CI Pipeline
|
||||
|
||||
#### Push and Verify
|
||||
|
||||
**Commit the workflow file:**
|
||||
```bash
|
||||
git add .github/workflows/test.yml
|
||||
git commit -m "ci: add automated test pipeline"
|
||||
git push
|
||||
```
|
||||
|
||||
**Watch the CI run:**
|
||||
- GitHub Actions: Go to Actions tab
|
||||
- GitLab CI: Go to CI/CD → Pipelines
|
||||
- Circle CI: Go to Pipelines
|
||||
|
||||
**Expected Result:**
|
||||
```
|
||||
✓ test (shard 1/4) - 3m 24s
|
||||
✓ test (shard 2/4) - 3m 18s
|
||||
✓ test (shard 3/4) - 3m 31s
|
||||
✓ test (shard 4/4) - 3m 15s
|
||||
✓ burn-in - 15m 42s
|
||||
```
|
||||
|
||||
#### Test on Pull Request
|
||||
|
||||
**Create test PR:**
|
||||
```bash
|
||||
git checkout -b test-ci-setup
|
||||
echo "# Test" > test.md
|
||||
git add test.md
|
||||
git commit -m "test: verify CI setup"
|
||||
git push -u origin test-ci-setup
|
||||
```
|
||||
|
||||
**Open PR and verify:**
|
||||
- Tests run automatically
|
||||
- Burn-in runs (if configured for PRs)
|
||||
- Selective tests run (if applicable)
|
||||
- All checks pass ✓
|
||||
|
||||
## What You Get
|
||||
|
||||
### Automated Test Execution
|
||||
- **On every PR** - Catch issues before merge
|
||||
- **On every push to main** - Protect production
|
||||
- **Nightly** - Comprehensive regression testing
|
||||
|
||||
### Parallel Execution
|
||||
- **4x faster feedback** - Shard across multiple workers
|
||||
- **Efficient resource usage** - Maximize CI runner utilization
|
||||
|
||||
### Selective Testing
|
||||
- **Run only affected tests** - Git diff-based selection
|
||||
- **Faster PR feedback** - Don't run entire suite every time
|
||||
|
||||
### Flakiness Detection
|
||||
- **Burn-in loops** - Run tests multiple times
|
||||
- **Early detection** - Catch flaky tests in PRs
|
||||
- **Confidence building** - Know tests are reliable
|
||||
|
||||
### Artifact Collection
|
||||
- **Test results** - Saved for 7 days
|
||||
- **Screenshots** - On test failures
|
||||
- **Videos** - Full test recordings
|
||||
- **Traces** - Playwright trace files for debugging
|
||||
|
||||
## Tips
|
||||
|
||||
### Start Simple, Add Complexity
|
||||
|
||||
**Week 1:** Basic pipeline
|
||||
```yaml
|
||||
- Run tests on PR
|
||||
- Single worker (no sharding)
|
||||
```
|
||||
|
||||
**Week 2:** Add parallelization
|
||||
```yaml
|
||||
- Shard across 4 workers
|
||||
- Faster feedback
|
||||
```
|
||||
|
||||
**Week 3:** Add selective testing
|
||||
```yaml
|
||||
- Git diff-based selection
|
||||
- Skip unaffected tests
|
||||
```
|
||||
|
||||
**Week 4:** Add burn-in
|
||||
```yaml
|
||||
- Detect flaky tests
|
||||
- Run on PR and nightly
|
||||
```
|
||||
|
||||
### Optimize for Feedback Speed
|
||||
|
||||
**Goal:** PR feedback in < 5 minutes
|
||||
|
||||
**Strategies:**
|
||||
- Shard tests across workers (4 workers = 4x faster)
|
||||
- Use selective testing (run 20% of tests, not 100%)
|
||||
- Cache dependencies (`actions/cache`, `cache: 'npm'`)
|
||||
- Run smoke tests first, full suite after
|
||||
|
||||
**Example fast workflow:**
|
||||
```yaml
|
||||
jobs:
|
||||
smoke:
|
||||
# Run critical path tests (2 min)
|
||||
run: npm run test:smoke
|
||||
|
||||
full:
|
||||
needs: smoke
|
||||
# Run full suite only if smoke passes (10 min)
|
||||
run: npm test
|
||||
```
|
||||
|
||||
### Use Test Tags
|
||||
|
||||
Tag tests for selective execution:
|
||||
|
||||
```typescript
|
||||
// Critical path tests (always run)
|
||||
test('@critical should login', async ({ page }) => { });
|
||||
|
||||
// Smoke tests (run first)
|
||||
test('@smoke should load homepage', async ({ page }) => { });
|
||||
|
||||
// Slow tests (run nightly only)
|
||||
test('@slow should process large file', async ({ page }) => { });
|
||||
|
||||
// Skip in CI
|
||||
test('@local-only should use local service', async ({ page }) => { });
|
||||
```
|
||||
|
||||
**In CI:**
|
||||
```bash
|
||||
# PR: Run critical and smoke only
|
||||
npx playwright test --grep "@critical|@smoke"
|
||||
|
||||
# Nightly: Run everything except local-only
|
||||
npx playwright test --grep-invert "@local-only"
|
||||
```
|
||||
|
||||
### Monitor CI Performance
|
||||
|
||||
Track metrics:
|
||||
|
||||
```markdown
|
||||
## CI Metrics
|
||||
|
||||
| Metric | Target | Current | Status |
|
||||
|--------|--------|---------|--------|
|
||||
| PR feedback time | < 5 min | 3m 24s | ✅ |
|
||||
| Full suite time | < 15 min | 12m 18s | ✅ |
|
||||
| Flakiness rate | < 1% | 0.3% | ✅ |
|
||||
| CI cost/month | < $100 | $75 | ✅ |
|
||||
```
|
||||
|
||||
### Handle Flaky Tests
|
||||
|
||||
When burn-in detects flakiness:
|
||||
|
||||
1. **Quarantine flaky test:**
|
||||
```typescript
|
||||
test.skip('flaky test - investigating', async ({ page }) => {
|
||||
// TODO: Fix flakiness
|
||||
});
|
||||
```
|
||||
|
||||
2. **Investigate with trace viewer:**
|
||||
```bash
|
||||
npx playwright show-trace test-results/trace.zip
|
||||
```
|
||||
|
||||
3. **Fix root cause:**
|
||||
- Add network-first patterns
|
||||
- Remove hard waits
|
||||
- Fix race conditions
|
||||
|
||||
4. **Verify fix:**
|
||||
```bash
|
||||
npm run test:burn-in -- tests/flaky.spec.ts --repeat 20
|
||||
```
|
||||
|
||||
### Secure Secrets
|
||||
|
||||
**Don't commit secrets to code:**
|
||||
```yaml
|
||||
# ❌ Bad
|
||||
- run: API_KEY=sk-1234... npm test
|
||||
|
||||
# ✅ Good
|
||||
- run: npm test
|
||||
env:
|
||||
API_KEY: ${{ secrets.API_KEY }}
|
||||
```
|
||||
|
||||
**Use environment-specific secrets:**
|
||||
- `STAGING_API_URL`
|
||||
- `PROD_API_URL`
|
||||
- `TEST_API_URL`
|
||||
|
||||
### Cache Aggressively
|
||||
|
||||
Speed up CI with caching:
|
||||
|
||||
```yaml
|
||||
# Cache node_modules
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
cache: 'npm'
|
||||
|
||||
# Cache Playwright browsers
|
||||
- name: Cache Playwright browsers
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/ms-playwright
|
||||
key: playwright-${{ hashFiles('package-lock.json') }}
|
||||
```
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Tests Pass Locally, Fail in CI
|
||||
|
||||
**Symptoms:**
|
||||
- Green locally, red in CI
|
||||
- "Works on my machine"
|
||||
|
||||
**Common Causes:**
|
||||
- Different Node version
|
||||
- Different browser version
|
||||
- Missing environment variables
|
||||
- Timezone differences
|
||||
- Race conditions (CI slower)
|
||||
|
||||
**Solutions:**
|
||||
```yaml
|
||||
# Pin Node version
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: '.nvmrc'
|
||||
|
||||
# Pin browser versions
|
||||
- run: npx playwright install --with-deps chromium@1.40.0
|
||||
|
||||
# Set timezone
|
||||
env:
|
||||
TZ: 'America/New_York'
|
||||
```
|
||||
|
||||
### CI Takes Too Long
|
||||
|
||||
**Problem:** CI takes 30+ minutes, developers wait too long.
|
||||
|
||||
**Solutions:**
|
||||
1. **Shard tests:** 4 workers = 4x faster
|
||||
2. **Selective testing:** Only run affected tests on PR
|
||||
3. **Smoke tests first:** Run critical path (2 min), full suite after
|
||||
4. **Cache dependencies:** `npm ci` with cache
|
||||
5. **Optimize tests:** Remove slow tests, hard waits
|
||||
|
||||
### Burn-In Always Fails
|
||||
|
||||
**Problem:** Burn-in job fails every time.
|
||||
|
||||
**Cause:** Test suite is flaky.
|
||||
|
||||
**Solution:**
|
||||
1. Identify flaky tests (check which iteration fails)
|
||||
2. Fix flaky tests using `*test-review`
|
||||
3. Re-run burn-in on specific files:
|
||||
```bash
|
||||
npm run test:burn-in tests/flaky.spec.ts
|
||||
```
|
||||
|
||||
### Out of CI Minutes
|
||||
|
||||
**Problem:** Using too many CI minutes, hitting plan limit.
|
||||
|
||||
**Solutions:**
|
||||
1. Run full suite only on main branch
|
||||
2. Use selective testing on PRs
|
||||
3. Run expensive tests nightly only
|
||||
4. Self-host runners (for GitHub Actions)
|
||||
|
||||
## Related Guides
|
||||
|
||||
- [How to Set Up Test Framework](/docs/how-to/workflows/setup-test-framework.md) - Run first
|
||||
- [How to Run Test Review](/docs/how-to/workflows/run-test-review.md) - Audit CI tests
|
||||
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) - Burn-in utility
|
||||
|
||||
## Understanding the Concepts
|
||||
|
||||
- [Test Quality Standards](/docs/explanation/tea/test-quality-standards.md) - Why determinism matters
|
||||
- [Network-First Patterns](/docs/explanation/tea/network-first-patterns.md) - Avoid CI flakiness
|
||||
|
||||
## Reference
|
||||
|
||||
- [Command: *ci](/docs/reference/tea/commands.md#ci) - Full command reference
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - CI-related config options
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -66,19 +66,18 @@ Type "exit" or "done" to conclude the session. Participating agents will say per
|
|||
|
||||
## Example Party Compositions
|
||||
|
||||
| Topic | Typical Agents |
|
||||
|-------|---------------|
|
||||
| **Product Strategy** | PM + Innovation Strategist (CIS) + Analyst |
|
||||
| **Technical Design** | Architect + Creative Problem Solver (CIS) + Game Architect |
|
||||
| **User Experience** | UX Designer + Design Thinking Coach (CIS) + Storyteller (CIS) |
|
||||
| **Quality Assessment** | TEA + DEV + Architect |
|
||||
| Topic | Typical Agents |
|
||||
| ---------------------- | ----------------------------------------------------- |
|
||||
| **Product Strategy** | PM + Innovation Strategist + Analyst |
|
||||
| **Technical Design** | Architect + Creative Problem Solver + Game Architect |
|
||||
| **User Experience** | UX Designer + Design Thinking Coach + Storyteller |
|
||||
| **Quality Assessment** | TEA + DEV + Architect |
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Intelligent agent selection** — Selects based on expertise needed
|
||||
- **Authentic personalities** — Each agent maintains their unique voice
|
||||
- **Natural cross-talk** — Agents reference and build on each other
|
||||
- **Optional TTS** — Voice configurations for each agent
|
||||
- **Graceful exit** — Personalized farewells
|
||||
|
||||
## Tips
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
title: "How to Set Up a Test Framework"
|
||||
title: "How to Set Up a Test Framework with TEA"
|
||||
description: How to set up a production-ready test framework using TEA
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -6,117 +6,154 @@ Terminology reference for the BMad Method.
|
|||
|
||||
## Core Concepts
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **Agent** | Specialized AI persona with specific expertise (PM, Architect, SM, DEV, TEA) that guides users through workflows and creates deliverables. |
|
||||
| **BMad** | Breakthrough Method of Agile AI Driven Development — AI-driven agile framework with specialized agents, guided workflows, and scale-adaptive intelligence. |
|
||||
| **BMad Method** | Complete methodology for AI-assisted software development, encompassing planning, architecture, implementation, and quality assurance workflows that adapt to project complexity. |
|
||||
| **BMM** | BMad Method Module — core orchestration system providing comprehensive lifecycle management through specialized agents and workflows. |
|
||||
| **Scale-Adaptive System** | Intelligent workflow orchestration that adjusts planning depth and documentation requirements based on project needs through three planning tracks. |
|
||||
| **Workflow** | Multi-step guided process that orchestrates AI agent activities to produce specific deliverables. Workflows are interactive and adapt to user context. |
|
||||
| Term | Definition |
|
||||
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Agent** | Specialized AI persona with specific expertise (PM, Architect, SM, DEV, TEA) that guides users through workflows and creates deliverables. |
|
||||
| **BMad** | Breakthrough Method of Agile AI-Driven Development — AI-driven agile framework with specialized agents, guided workflows, and scale-adaptive intelligence. |
|
||||
| **BMad Method** | Complete methodology for AI-assisted software development, encompassing planning, architecture, implementation, and quality assurance workflows that adapt to project complexity. |
|
||||
| **BMM** | BMad Method Module — core orchestration system providing comprehensive lifecycle management through specialized agents and workflows. |
|
||||
| **Scale-Adaptive System** | Intelligent workflow orchestration that adjusts planning depth and documentation requirements based on project needs through three planning tracks. |
|
||||
| **Workflow** | Multi-step guided process that orchestrates AI agent activities to produce specific deliverables. Workflows are interactive and adapt to user context. |
|
||||
|
||||
## Scale and Complexity
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **BMad Method Track** | Full product planning track using PRD + Architecture + UX. Best for products, platforms, and complex features. Typical range: 10-50+ stories. |
|
||||
| Term | Definition |
|
||||
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **BMad Method Track** | Full product planning track using PRD + Architecture + UX. Best for products, platforms, and complex features. Typical range: 10-50+ stories. |
|
||||
| **Enterprise Method Track** | Extended planning track adding Security Architecture, DevOps Strategy, and Test Strategy. Best for compliance needs and multi-tenant systems. Typical range: 30+ stories. |
|
||||
| **Planning Track** | Methodology path (Quick Flow, BMad Method, or Enterprise) chosen based on planning needs and complexity, not story count alone. |
|
||||
| **Quick Flow Track** | Fast implementation track using tech-spec only. Best for bug fixes, small features, and clear-scope changes. Typical range: 1-15 stories. |
|
||||
| **Planning Track** | Methodology path (Quick Flow, BMad Method, or Enterprise) chosen based on planning needs and complexity, not story count alone. |
|
||||
| **Quick Flow Track** | Fast implementation track using tech-spec only. Best for bug fixes, small features, and clear-scope changes. Typical range: 1-15 stories. |
|
||||
|
||||
## Planning Documents
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| Term | Definition |
|
||||
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Architecture Document** | *BMad Method/Enterprise.* System-wide design document defining structure, components, data models, integration patterns, security, and deployment. |
|
||||
| **Epics** | High-level feature groupings containing multiple related stories. Typically 5-15 stories each representing cohesive functionality. |
|
||||
| **Game Brief** | *BMGD.* Document capturing game's core vision, pillars, target audience, and scope. Foundation for the GDD. |
|
||||
| **GDD** | *BMGD.* Game Design Document — comprehensive document detailing all aspects of game design: mechanics, systems, content, and more. |
|
||||
| **PRD** | *BMad Method/Enterprise.* Product Requirements Document containing vision, goals, FRs, NFRs, and success criteria. Focuses on WHAT to build. |
|
||||
| **Product Brief** | *Phase 1.* Optional strategic document capturing product vision, market context, and high-level requirements before detailed planning. |
|
||||
| **Tech-Spec** | *Quick Flow only.* Comprehensive technical plan with problem statement, solution approach, file-level changes, and testing strategy. |
|
||||
| **Epics** | High-level feature groupings containing multiple related stories. Typically 5-15 stories each representing cohesive functionality. |
|
||||
| **Game Brief** | *BMGD.* Document capturing game's core vision, pillars, target audience, and scope. Foundation for the GDD. |
|
||||
| **GDD** | *BMGD.* Game Design Document — comprehensive document detailing all aspects of game design: mechanics, systems, content, and more. |
|
||||
| **PRD** | *BMad Method/Enterprise.* Product Requirements Document containing vision, goals, FRs, NFRs, and success criteria. Focuses on WHAT to build. |
|
||||
| **Product Brief** | *Phase 1.* Optional strategic document capturing product vision, market context, and high-level requirements before detailed planning. |
|
||||
| **Tech-Spec** | *Quick Flow only.* Comprehensive technical plan with problem statement, solution approach, file-level changes, and testing strategy. |
|
||||
|
||||
## Workflow and Phases
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **Phase 0: Documentation** | *Brownfield.* Conditional prerequisite phase creating codebase documentation before planning. Only required if existing docs are insufficient. |
|
||||
| **Phase 1: Analysis** | Discovery phase including brainstorming, research, and product brief creation. Optional for Quick Flow, recommended for BMad Method. |
|
||||
| **Phase 2: Planning** | Required phase creating formal requirements. Routes to tech-spec (Quick Flow) or PRD (BMad Method/Enterprise). |
|
||||
| **Phase 3: Solutioning** | *BMad Method/Enterprise.* Architecture design phase including creation, validation, and gate checks. |
|
||||
| **Phase 4: Implementation** | Required sprint-based development through story-by-story iteration using sprint-planning, create-story, dev-story, and code-review workflows. |
|
||||
| **Quick Spec Flow** | Fast-track workflow for Quick Flow projects going straight from idea to tech-spec to implementation. |
|
||||
| **Workflow Init** | Initialization workflow creating bmm-workflow-status.yaml, detecting project type, and determining planning track. |
|
||||
| **Workflow Status** | Universal entry point checking for existing status file, displaying progress, and recommending next action. |
|
||||
| Term | Definition |
|
||||
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Phase 0: Documentation** | *Brownfield.* Conditional prerequisite phase creating codebase documentation before planning. Only required if existing docs are insufficient. |
|
||||
| **Phase 1: Analysis** | Discovery phase including brainstorming, research, and product brief creation. Optional for Quick Flow, recommended for BMad Method. |
|
||||
| **Phase 2: Planning** | Required phase creating formal requirements. Routes to tech-spec (Quick Flow) or PRD (BMad Method/Enterprise). |
|
||||
| **Phase 3: Solutioning** | *BMad Method/Enterprise.* Architecture design phase including creation, validation, and gate checks. |
|
||||
| **Phase 4: Implementation** | Required sprint-based development through story-by-story iteration using sprint-planning, create-story, dev-story, and code-review workflows. |
|
||||
| **Quick Spec Flow** | Fast-track workflow for Quick Flow projects going straight from idea to tech-spec to implementation. |
|
||||
| **Workflow Init** | Initialization workflow creating bmm-workflow-status.yaml, detecting project type, and determining planning track. |
|
||||
| **Workflow Status** | Universal entry point checking for existing status file, displaying progress, and recommending next action. |
|
||||
|
||||
## Agents and Roles
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **Analyst** | Agent that initializes workflows, conducts research, creates product briefs, and tracks progress. Often the entry point for new projects. |
|
||||
| **Architect** | Agent designing system architecture, creating architecture documents, and validating designs. Primary agent for Phase 3. |
|
||||
| **BMad Master** | Meta-level orchestrator from BMad Core facilitating party mode and providing high-level guidance across all modules. |
|
||||
| **DEV** | Developer agent implementing stories, writing code, running tests, and performing code reviews. Primary implementer in Phase 4. |
|
||||
| **Game Architect** | *BMGD.* Agent designing game system architecture and validating game-specific technical designs. |
|
||||
| **Game Designer** | *BMGD.* Agent creating game design documents (GDD) and running game-specific workflows. |
|
||||
| **Party Mode** | Multi-agent collaboration feature where agents discuss challenges together. BMad Master orchestrates, selecting 2-3 relevant agents per message. |
|
||||
| **PM** | Product Manager agent creating PRDs and tech-specs. Primary agent for Phase 2 planning. |
|
||||
| **SM** | Scrum Master agent managing sprints, creating stories, and coordinating implementation. Primary orchestrator for Phase 4. |
|
||||
| **TEA** | Test Architect agent responsible for test strategy, quality gates, and NFR assessment. Integrates throughout all phases. |
|
||||
| **Technical Writer** | Agent specialized in creating technical documentation, diagrams, and maintaining documentation standards. |
|
||||
| **UX Designer** | Agent creating UX design documents, interaction patterns, and visual specifications for UI-heavy projects. |
|
||||
| Term | Definition |
|
||||
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| **Analyst** | Agent that initializes workflows, conducts research, creates product briefs, and tracks progress. Often the entry point for new projects. |
|
||||
| **Architect** | Agent designing system architecture, creating architecture documents, and validating designs. Primary agent for Phase 3. |
|
||||
| **BMad Master** | Meta-level orchestrator from BMad Core facilitating party mode and providing high-level guidance across all modules. |
|
||||
| **DEV** | Developer agent implementing stories, writing code, running tests, and performing code reviews. Primary implementer in Phase 4. |
|
||||
| **Game Architect** | *BMGD.* Agent designing game system architecture and validating game-specific technical designs. |
|
||||
| **Game Designer** | *BMGD.* Agent creating game design documents (GDD) and running game-specific workflows. |
|
||||
| **Party Mode** | Multi-agent collaboration feature where agents discuss challenges together. BMad Master orchestrates, selecting 2-3 relevant agents per message. |
|
||||
| **PM** | Product Manager agent creating PRDs and tech-specs. Primary agent for Phase 2 planning. |
|
||||
| **SM** | Scrum Master agent managing sprints, creating stories, and coordinating implementation. Primary orchestrator for Phase 4. |
|
||||
| **TEA** | Test Architect agent responsible for test strategy, quality gates, and NFR assessment. Integrates throughout all phases. |
|
||||
| **Technical Writer** | Agent specialized in creating technical documentation, diagrams, and maintaining documentation standards. |
|
||||
| **UX Designer** | Agent creating UX design documents, interaction patterns, and visual specifications for UI-heavy projects. |
|
||||
|
||||
## Status and Tracking
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **bmm-workflow-status.yaml** | *Phases 1-3.* Tracking file showing current phase, completed workflows, and next recommended actions. |
|
||||
| **DoD** | Definition of Done — criteria for marking a story complete: implementation done, tests passing, code reviewed, docs updated. |
|
||||
| **Epic Status Progression** | `backlog → in-progress → done` — lifecycle states for epics during implementation. |
|
||||
| **Gate Check** | Validation workflow (implementation-readiness) ensuring PRD, Architecture, and Epics are aligned before Phase 4. |
|
||||
| **Retrospective** | Workflow after each epic capturing learnings and improvements for continuous improvement. |
|
||||
| **sprint-status.yaml** | *Phase 4.* Single source of truth for implementation tracking containing all epics, stories, and their statuses. |
|
||||
| **Story Status Progression** | `backlog → ready-for-dev → in-progress → review → done` — lifecycle states for stories. |
|
||||
| Term | Definition |
|
||||
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **bmm-workflow-status.yaml** | *Phases 1-3.* Tracking file showing current phase, completed workflows, and next recommended actions. |
|
||||
| **DoD** | Definition of Done — criteria for marking a story complete: implementation done, tests passing, code reviewed, docs updated. |
|
||||
| **Epic Status Progression** | `backlog → in-progress → done` — lifecycle states for epics during implementation. |
|
||||
| **Gate Check** | Validation workflow (implementation-readiness) ensuring PRD, Architecture, and Epics are aligned before Phase 4. |
|
||||
| **Retrospective** | Workflow after each epic capturing learnings and improvements for continuous improvement. |
|
||||
| **sprint-status.yaml** | *Phase 4.* Single source of truth for implementation tracking containing all epics, stories, and their statuses. |
|
||||
| **Story Status Progression** | `backlog → ready-for-dev → in-progress → review → done` — lifecycle states for stories. |
|
||||
|
||||
## Project Types
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **Brownfield** | Existing project with established codebase and patterns. Requires understanding existing architecture and planning integration. |
|
||||
| **Convention Detection** | *Quick Flow.* Feature auto-detecting existing code style, naming conventions, and frameworks from brownfield codebases. |
|
||||
| **document-project** | *Brownfield.* Workflow analyzing and documenting existing codebase with three scan levels: quick, deep, exhaustive. |
|
||||
| **Feature Flags** | *Brownfield.* Implementation technique for gradual rollout, easy rollback, and A/B testing of new functionality. |
|
||||
| **Greenfield** | New project starting from scratch with freedom to establish patterns, choose stack, and design from clean slate. |
|
||||
| **Integration Points** | *Brownfield.* Specific locations where new code connects with existing systems. Must be documented in tech-specs. |
|
||||
| Term | Definition |
|
||||
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Brownfield** | Existing project with established codebase and patterns. Requires understanding existing architecture and planning integration. |
|
||||
| **Convention Detection** | *Quick Flow.* Feature auto-detecting existing code style, naming conventions, and frameworks from brownfield codebases. |
|
||||
| **document-project** | *Brownfield.* Workflow analyzing and documenting existing codebase with three scan levels: quick, deep, exhaustive. |
|
||||
| **Feature Flags** | *Brownfield.* Implementation technique for gradual rollout, easy rollback, and A/B testing of new functionality. |
|
||||
| **Greenfield** | New project starting from scratch with freedom to establish patterns, choose stack, and design from clean slate. |
|
||||
| **Integration Points** | *Brownfield.* Specific locations where new code connects with existing systems. Must be documented in tech-specs. |
|
||||
|
||||
## Implementation Terms
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| Term | Definition |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| **Context Engineering** | Loading domain-specific standards into AI context automatically via manifests, ensuring consistent outputs regardless of prompt variation. |
|
||||
| **Correct Course** | Workflow for navigating significant changes when implementation is off-track. Analyzes impact and recommends adjustments. |
|
||||
| **Shard / Sharding** | Splitting large planning documents into section-based files for LLM optimization. Phase 4 workflows load only needed sections. |
|
||||
| **Sprint** | Time-boxed period of development work, typically 1-2 weeks. |
|
||||
| **Sprint Planning** | Workflow initializing Phase 4 by creating sprint-status.yaml and extracting epics/stories from planning docs. |
|
||||
| **Story** | Single unit of implementable work with clear acceptance criteria, typically 2-8 hours of effort. Grouped into epics. |
|
||||
| **Story Context** | Implementation guidance embedded in story files during create-story, referencing existing patterns and approaches. |
|
||||
| **Story File** | Markdown file containing story description, acceptance criteria, technical notes, and testing requirements. |
|
||||
| **Track Selection** | Automatic analysis by workflow-init suggesting appropriate track based on complexity indicators. User can override. |
|
||||
| **Correct Course** | Workflow for navigating significant changes when implementation is off-track. Analyzes impact and recommends adjustments. |
|
||||
| **Shard / Sharding** | Splitting large planning documents into section-based files for LLM optimization. Phase 4 workflows load only needed sections. |
|
||||
| **Sprint** | Time-boxed period of development work, typically 1-2 weeks. |
|
||||
| **Sprint Planning** | Workflow initializing Phase 4 by creating sprint-status.yaml and extracting epics/stories from planning docs. |
|
||||
| **Story** | Single unit of implementable work with clear acceptance criteria, typically 2-8 hours of effort. Grouped into epics. |
|
||||
| **Story Context** | Implementation guidance embedded in story files during create-story, referencing existing patterns and approaches. |
|
||||
| **Story File** | Markdown file containing story description, acceptance criteria, technical notes, and testing requirements. |
|
||||
| **Track Selection** | Automatic analysis by workflow-init suggesting appropriate track based on complexity indicators. User can override. |
|
||||
|
||||
## Game Development Terms
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **Core Fantasy** | *BMGD.* The emotional experience players seek from your game — what they want to FEEL. |
|
||||
| **Core Loop** | *BMGD.* Fundamental cycle of actions players repeat throughout gameplay. The heart of your game. |
|
||||
| **Design Pillar** | *BMGD.* Core principle guiding all design decisions. Typically 3-5 pillars define a game's identity. |
|
||||
| **Environmental Storytelling** | *BMGD.* Narrative communicated through the game world itself rather than explicit dialogue. |
|
||||
| **Game Type** | *BMGD.* Genre classification determining which specialized GDD sections are included. |
|
||||
| **MDA Framework** | *BMGD.* Mechanics → Dynamics → Aesthetics — framework for analyzing and designing games. |
|
||||
| **Meta-Progression** | *BMGD.* Persistent progression carrying between individual runs or sessions. |
|
||||
| **Metroidvania** | *BMGD.* Genre featuring interconnected world exploration with ability-gated progression. |
|
||||
| **Narrative Complexity** | *BMGD.* How central story is to the game: Critical, Heavy, Moderate, or Light. |
|
||||
| **Permadeath** | *BMGD.* Game mechanic where character death is permanent, typically requiring a new run. |
|
||||
| **Player Agency** | *BMGD.* Degree to which players can make meaningful choices affecting outcomes. |
|
||||
| **Procedural Generation** | *BMGD.* Algorithmic creation of game content (levels, items, characters) rather than hand-crafted. |
|
||||
| **Roguelike** | *BMGD.* Genre featuring procedural generation, permadeath, and run-based progression. |
|
||||
| Term | Definition |
|
||||
| ------------------------------ | ---------------------------------------------------------------------------------------------------- |
|
||||
| **Core Fantasy** | *BMGD.* The emotional experience players seek from your game — what they want to FEEL. |
|
||||
| **Core Loop** | *BMGD.* Fundamental cycle of actions players repeat throughout gameplay. The heart of your game. |
|
||||
| **Design Pillar** | *BMGD.* Core principle guiding all design decisions. Typically 3-5 pillars define a game's identity. |
|
||||
| **Environmental Storytelling** | *BMGD.* Narrative communicated through the game world itself rather than explicit dialogue. |
|
||||
| **Game Type** | *BMGD.* Genre classification determining which specialized GDD sections are included. |
|
||||
| **MDA Framework** | *BMGD.* Mechanics → Dynamics → Aesthetics — framework for analyzing and designing games. |
|
||||
| **Meta-Progression** | *BMGD.* Persistent progression carrying between individual runs or sessions. |
|
||||
| **Metroidvania** | *BMGD.* Genre featuring interconnected world exploration with ability-gated progression. |
|
||||
| **Narrative Complexity** | *BMGD.* How central story is to the game: Critical, Heavy, Moderate, or Light. |
|
||||
| **Permadeath** | *BMGD.* Game mechanic where character death is permanent, typically requiring a new run. |
|
||||
| **Player Agency** | *BMGD.* Degree to which players can make meaningful choices affecting outcomes. |
|
||||
| **Procedural Generation** | *BMGD.* Algorithmic creation of game content (levels, items, characters) rather than hand-crafted. |
|
||||
| **Roguelike** | *BMGD.* Genre featuring procedural generation, permadeath, and run-based progression. |
|
||||
|
||||
## Test Architect (TEA) Concepts
|
||||
|
||||
| Term | Definition |
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **ATDD** | Acceptance Test-Driven Development — Generating failing acceptance tests BEFORE implementation (TDD red phase). |
|
||||
| **Burn-in Testing** | Running tests multiple times (typically 5-10 iterations) to detect flakiness and intermittent failures. |
|
||||
| **Component Testing** | Testing UI components in isolation using framework-specific tools (Cypress Component Testing or Vitest + React Testing Library). |
|
||||
| **Coverage Traceability** | Mapping acceptance criteria to implemented tests with classification (FULL/PARTIAL/NONE) to identify gaps and measure completeness. |
|
||||
| **Epic-Level Test Design** | Test planning per epic (Phase 4) focusing on risk assessment, priorities, and coverage strategy for that specific epic. |
|
||||
| **Fixture Architecture** | Pattern of building pure functions first, then wrapping in framework-specific fixtures for testability, reusability, and composition. |
|
||||
| **Gate Decision** | Go/no-go decision for release with four outcomes: PASS ✅ (ready), CONCERNS ⚠️ (proceed with mitigation), FAIL ❌ (blocked), WAIVED ⏭️ (approved despite issues). |
|
||||
| **Knowledge Fragment** | Individual markdown file in TEA's knowledge base covering a specific testing pattern or practice (33 fragments total). |
|
||||
| **MCP Enhancements** | Model Context Protocol servers enabling live browser verification during test generation (exploratory, recording, and healing modes). |
|
||||
| **Network-First Pattern** | Testing pattern that waits for actual network responses instead of fixed timeouts to avoid race conditions and flakiness. |
|
||||
| **NFR Assessment** | Validation of non-functional requirements (security, performance, reliability, maintainability) with evidence-based decisions. |
|
||||
| **Playwright Utils** | Optional package (`@seontechnologies/playwright-utils`) providing production-ready fixtures and utilities for Playwright tests. |
|
||||
| **Risk-Based Testing** | Testing approach where depth scales with business impact using probability × impact scoring (1-9 scale). |
|
||||
| **System-Level Test Design** | Test planning at architecture level (Phase 3) focusing on testability review, ADR mapping, and test infrastructure needs. |
|
||||
| **tea-index.csv** | Manifest file tracking all knowledge fragments, their descriptions, tags, and which workflows load them. |
|
||||
| **TEA Integrated** | Full BMad Method integration with TEA workflows across all phases (Phase 2, 3, 4, and Release Gate). |
|
||||
| **TEA Lite** | Beginner approach using just `*automate` workflow to test existing features (simplest way to use TEA). |
|
||||
| **TEA Solo** | Standalone engagement model using TEA without full BMad Method integration (bring your own requirements). |
|
||||
| **Test Priorities** | Classification system for test importance: P0 (critical path), P1 (high value), P2 (medium value), P3 (low value). |
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Complete TEA capabilities
|
||||
- [TEA Knowledge Base](/docs/reference/tea/knowledge-base.md) - Fragment index
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - Workflow reference
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Config options
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,254 @@
|
|||
---
|
||||
title: "TEA Command Reference"
|
||||
description: Quick reference for all 8 TEA workflows - inputs, outputs, and links to detailed guides
|
||||
---
|
||||
|
||||
# TEA Command Reference
|
||||
|
||||
Quick reference for all 8 TEA (Test Architect) workflows. For detailed step-by-step guides, see the how-to documentation.
|
||||
|
||||
## Quick Index
|
||||
|
||||
- [*framework](#framework) - Scaffold test framework
|
||||
- [*ci](#ci) - Setup CI/CD pipeline
|
||||
- [*test-design](#test-design) - Risk-based test planning
|
||||
- [*atdd](#atdd) - Acceptance TDD
|
||||
- [*automate](#automate) - Test automation
|
||||
- [*test-review](#test-review) - Quality audit
|
||||
- [*nfr-assess](#nfr-assess) - NFR assessment
|
||||
- [*trace](#trace) - Coverage traceability
|
||||
|
||||
---
|
||||
|
||||
## *framework
|
||||
|
||||
**Purpose:** Scaffold production-ready test framework (Playwright or Cypress)
|
||||
|
||||
**Phase:** Phase 3 (Solutioning)
|
||||
|
||||
**Frequency:** Once per project
|
||||
|
||||
**Key Inputs:**
|
||||
- Tech stack, test framework choice, testing scope
|
||||
|
||||
**Key Outputs:**
|
||||
- `tests/` directory with `support/fixtures/` and `support/helpers/`
|
||||
- `playwright.config.ts` or `cypress.config.ts`
|
||||
- `.env.example`, `.nvmrc`
|
||||
- Sample tests with best practices
|
||||
|
||||
**How-To Guide:** [Setup Test Framework](/docs/how-to/workflows/setup-test-framework.md)
|
||||
|
||||
---
|
||||
|
||||
## *ci
|
||||
|
||||
**Purpose:** Setup CI/CD pipeline with selective testing and burn-in
|
||||
|
||||
**Phase:** Phase 3 (Solutioning)
|
||||
|
||||
**Frequency:** Once per project
|
||||
|
||||
**Key Inputs:**
|
||||
- CI platform (GitHub Actions, GitLab CI, etc.)
|
||||
- Sharding strategy, burn-in preferences
|
||||
|
||||
**Key Outputs:**
|
||||
- Platform-specific CI workflow (`.github/workflows/test.yml`, etc.)
|
||||
- Parallel execution configuration
|
||||
- Burn-in loops for flakiness detection
|
||||
- Secrets checklist
|
||||
|
||||
**How-To Guide:** [Setup CI Pipeline](/docs/how-to/workflows/setup-ci.md)
|
||||
|
||||
---
|
||||
|
||||
## *test-design
|
||||
|
||||
**Purpose:** Risk-based test planning with coverage strategy
|
||||
|
||||
**Phase:** Phase 3 (system-level), Phase 4 (epic-level)
|
||||
|
||||
**Frequency:** Once (system), per epic (epic-level)
|
||||
|
||||
**Modes:**
|
||||
- **System-level:** Architecture testability review
|
||||
- **Epic-level:** Per-epic risk assessment
|
||||
|
||||
**Key Inputs:**
|
||||
- Architecture/epic, requirements, ADRs
|
||||
|
||||
**Key Outputs:**
|
||||
- `test-design-system.md` or `test-design-epic-N.md`
|
||||
- Risk assessment (probability × impact scores)
|
||||
- Test priorities (P0-P3)
|
||||
- Coverage strategy
|
||||
|
||||
**MCP Enhancement:** Exploratory mode (live browser UI discovery)
|
||||
|
||||
**How-To Guide:** [Run Test Design](/docs/how-to/workflows/run-test-design.md)
|
||||
|
||||
---
|
||||
|
||||
## *atdd
|
||||
|
||||
**Purpose:** Generate failing acceptance tests BEFORE implementation (TDD red phase)
|
||||
|
||||
**Phase:** Phase 4 (Implementation)
|
||||
|
||||
**Frequency:** Per story (optional)
|
||||
|
||||
**Key Inputs:**
|
||||
- Story with acceptance criteria, test design, test levels
|
||||
|
||||
**Key Outputs:**
|
||||
- Failing tests (`tests/api/`, `tests/e2e/`)
|
||||
- Implementation checklist
|
||||
- All tests fail initially (red phase)
|
||||
|
||||
**MCP Enhancement:** Recording mode (for skeleton UI only - rare)
|
||||
|
||||
**How-To Guide:** [Run ATDD](/docs/how-to/workflows/run-atdd.md)
|
||||
|
||||
---
|
||||
|
||||
## *automate
|
||||
|
||||
**Purpose:** Expand test coverage after implementation
|
||||
|
||||
**Phase:** Phase 4 (Implementation)
|
||||
|
||||
**Frequency:** Per story/feature
|
||||
|
||||
**Key Inputs:**
|
||||
- Feature description, test design, existing tests to avoid duplication
|
||||
|
||||
**Key Outputs:**
|
||||
- Comprehensive test suite (`tests/e2e/`, `tests/api/`)
|
||||
- Updated fixtures, README
|
||||
- Definition of Done summary
|
||||
|
||||
**MCP Enhancement:** Healing + Recording modes (fix tests, verify selectors)
|
||||
|
||||
**How-To Guide:** [Run Automate](/docs/how-to/workflows/run-automate.md)
|
||||
|
||||
---
|
||||
|
||||
## *test-review
|
||||
|
||||
**Purpose:** Audit test quality with 0-100 scoring
|
||||
|
||||
**Phase:** Phase 4 (optional per story), Release Gate
|
||||
|
||||
**Frequency:** Per epic or before release
|
||||
|
||||
**Key Inputs:**
|
||||
- Test scope (file, directory, or entire suite)
|
||||
|
||||
**Key Outputs:**
|
||||
- `test-review.md` with quality score (0-100)
|
||||
- Critical issues with fixes
|
||||
- Recommendations
|
||||
- Category scores (Determinism, Isolation, Assertions, Structure, Performance)
|
||||
|
||||
**Scoring Categories:**
|
||||
- Determinism: 35 points
|
||||
- Isolation: 25 points
|
||||
- Assertions: 20 points
|
||||
- Structure: 10 points
|
||||
- Performance: 10 points
|
||||
|
||||
**How-To Guide:** [Run Test Review](/docs/how-to/workflows/run-test-review.md)
|
||||
|
||||
---
|
||||
|
||||
## *nfr-assess
|
||||
|
||||
**Purpose:** Validate non-functional requirements with evidence
|
||||
|
||||
**Phase:** Phase 2 (enterprise), Release Gate
|
||||
|
||||
**Frequency:** Per release (enterprise projects)
|
||||
|
||||
**Key Inputs:**
|
||||
- NFR categories (Security, Performance, Reliability, Maintainability)
|
||||
- Thresholds, evidence location
|
||||
|
||||
**Key Outputs:**
|
||||
- `nfr-assessment.md`
|
||||
- Category assessments (PASS/CONCERNS/FAIL)
|
||||
- Mitigation plans
|
||||
- Gate decision inputs
|
||||
|
||||
**How-To Guide:** [Run NFR Assessment](/docs/how-to/workflows/run-nfr-assess.md)
|
||||
|
||||
---
|
||||
|
||||
## *trace
|
||||
|
||||
**Purpose:** Requirements traceability + quality gate decision
|
||||
|
||||
**Phase:** Phase 2/4 (traceability), Release Gate (decision)
|
||||
|
||||
**Frequency:** Baseline, per epic refresh, release gate
|
||||
|
||||
**Two-Phase Workflow:**
|
||||
|
||||
**Phase 1: Traceability**
|
||||
- Requirements → test mapping
|
||||
- Coverage classification (FULL/PARTIAL/NONE)
|
||||
- Gap prioritization
|
||||
- Output: `traceability-matrix.md`
|
||||
|
||||
**Phase 2: Gate Decision**
|
||||
- PASS/CONCERNS/FAIL/WAIVED decision
|
||||
- Evidence-based (coverage %, quality scores, NFRs)
|
||||
- Output: `gate-decision-{gate_type}-{story_id}.md`
|
||||
|
||||
**Gate Rules:**
|
||||
- P0 coverage: 100% required
|
||||
- P1 coverage: ≥90% for PASS, 80-89% for CONCERNS, <80% FAIL
|
||||
- Overall coverage: ≥80% required
|
||||
|
||||
**How-To Guide:** [Run Trace](/docs/how-to/workflows/run-trace.md)
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| Command | Phase | Frequency | Primary Output |
|
||||
|---------|-------|-----------|----------------|
|
||||
| `*framework` | 3 | Once | Test infrastructure |
|
||||
| `*ci` | 3 | Once | CI/CD pipeline |
|
||||
| `*test-design` | 3, 4 | System + per epic | Test design doc |
|
||||
| `*atdd` | 4 | Per story (optional) | Failing tests |
|
||||
| `*automate` | 4 | Per story | Passing tests |
|
||||
| `*test-review` | 4, Gate | Per epic/release | Quality report |
|
||||
| `*nfr-assess` | 2, Gate | Per release | NFR assessment |
|
||||
| `*trace` | 2, 4, Gate | Baseline + refresh + gate | Coverage matrix + decision |
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
**How-To Guides (Detailed Instructions):**
|
||||
- [Setup Test Framework](/docs/how-to/workflows/setup-test-framework.md)
|
||||
- [Setup CI Pipeline](/docs/how-to/workflows/setup-ci.md)
|
||||
- [Run Test Design](/docs/how-to/workflows/run-test-design.md)
|
||||
- [Run ATDD](/docs/how-to/workflows/run-atdd.md)
|
||||
- [Run Automate](/docs/how-to/workflows/run-automate.md)
|
||||
- [Run Test Review](/docs/how-to/workflows/run-test-review.md)
|
||||
- [Run NFR Assessment](/docs/how-to/workflows/run-nfr-assess.md)
|
||||
- [Run Trace](/docs/how-to/workflows/run-trace.md)
|
||||
|
||||
**Explanation:**
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Complete TEA lifecycle
|
||||
- [Engagement Models](/docs/explanation/tea/engagement-models.md) - When to use which workflows
|
||||
|
||||
**Reference:**
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Config options
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md) - Pattern fragments
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,678 @@
|
|||
---
|
||||
title: "TEA Configuration Reference"
|
||||
description: Complete reference for TEA configuration options and file locations
|
||||
---
|
||||
|
||||
# TEA Configuration Reference
|
||||
|
||||
Complete reference for all TEA (Test Architect) configuration options.
|
||||
|
||||
## Configuration File Locations
|
||||
|
||||
### User Configuration (Installer-Generated)
|
||||
|
||||
**Location:** `_bmad/bmm/config.yaml`
|
||||
|
||||
**Purpose:** Project-specific configuration values for your repository
|
||||
|
||||
**Created By:** BMad installer
|
||||
|
||||
**Status:** Typically gitignored (user-specific values)
|
||||
|
||||
**Usage:** Edit this file to change TEA behavior in your project
|
||||
|
||||
**Example:**
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml
|
||||
project_name: my-awesome-app
|
||||
user_skill_level: intermediate
|
||||
output_folder: _bmad-output
|
||||
tea_use_playwright_utils: true
|
||||
tea_use_mcp_enhancements: false
|
||||
```
|
||||
|
||||
### Canonical Schema (Source of Truth)
|
||||
|
||||
**Location:** `src/modules/bmm/module.yaml`
|
||||
|
||||
**Purpose:** Defines available configuration keys, defaults, and installer prompts
|
||||
|
||||
**Created By:** BMAD maintainers (part of BMAD repo)
|
||||
|
||||
**Status:** Versioned in BMAD repository
|
||||
|
||||
**Usage:** Reference only (do not edit unless contributing to BMAD)
|
||||
|
||||
**Note:** The installer reads `module.yaml` to prompt for config values, then writes user choices to `_bmad/bmm/config.yaml` in your project.
|
||||
|
||||
---
|
||||
|
||||
## TEA Configuration Options
|
||||
|
||||
### tea_use_playwright_utils
|
||||
|
||||
Enable Playwright Utils integration for production-ready fixtures and utilities.
|
||||
|
||||
**Schema Location:** `src/modules/bmm/module.yaml:52-56`
|
||||
|
||||
**User Config:** `_bmad/bmm/config.yaml`
|
||||
|
||||
**Type:** `boolean`
|
||||
|
||||
**Default:** `false` (set via installer prompt during installation)
|
||||
|
||||
**Installer Prompt:**
|
||||
```
|
||||
Are you using playwright-utils (@seontechnologies/playwright-utils) in your project?
|
||||
You must install packages yourself, or use test architect's *framework command.
|
||||
```
|
||||
|
||||
**Purpose:** Enables TEA to:
|
||||
- Include playwright-utils in `*framework` scaffold
|
||||
- Generate tests using playwright-utils fixtures
|
||||
- Review tests against playwright-utils patterns
|
||||
- Configure CI with burn-in and selective testing utilities
|
||||
|
||||
**Affects Workflows:**
|
||||
- `*framework` - Includes playwright-utils imports and fixture examples
|
||||
- `*atdd` - Uses fixtures like `apiRequest`, `authSession` in generated tests
|
||||
- `*automate` - Leverages utilities for test patterns
|
||||
- `*test-review` - Reviews against playwright-utils best practices
|
||||
- `*ci` - Includes burn-in utility and selective testing
|
||||
|
||||
**Example (Enable):**
|
||||
```yaml
|
||||
tea_use_playwright_utils: true
|
||||
```
|
||||
|
||||
**Example (Disable):**
|
||||
```yaml
|
||||
tea_use_playwright_utils: false
|
||||
```
|
||||
|
||||
**Prerequisites:**
|
||||
```bash
|
||||
npm install -D @seontechnologies/playwright-utils
|
||||
```
|
||||
|
||||
**Related:**
|
||||
- [Integrate Playwright Utils Guide](/docs/how-to/customization/integrate-playwright-utils.md)
|
||||
- [Playwright Utils on npm](https://www.npmjs.com/package/@seontechnologies/playwright-utils)
|
||||
|
||||
---
|
||||
|
||||
### tea_use_mcp_enhancements
|
||||
|
||||
Enable Playwright MCP servers for live browser verification during test generation.
|
||||
|
||||
**Schema Location:** `src/modules/bmm/module.yaml:47-50`
|
||||
|
||||
**User Config:** `_bmad/bmm/config.yaml`
|
||||
|
||||
**Type:** `boolean`
|
||||
|
||||
**Default:** `false`
|
||||
|
||||
**Installer Prompt:**
|
||||
```
|
||||
Test Architect Playwright MCP capabilities (healing, exploratory, verification) are optionally available.
|
||||
You will have to setup your MCPs yourself; refer to https://docs.bmad-method.org/explanation/features/tea-overview for configuration examples.
|
||||
Would you like to enable MCP enhancements in Test Architect?
|
||||
```
|
||||
|
||||
**Purpose:** Enables TEA to use Model Context Protocol servers for:
|
||||
- Live browser automation during test design
|
||||
- Selector verification with actual DOM
|
||||
- Interactive UI discovery
|
||||
- Visual debugging and healing
|
||||
|
||||
**Affects Workflows:**
|
||||
- `*test-design` - Enables exploratory mode (browser-based UI discovery)
|
||||
- `*atdd` - Enables recording mode (verify selectors with live browser)
|
||||
- `*automate` - Enables healing mode (fix tests with visual debugging)
|
||||
|
||||
**MCP Servers Required:**
|
||||
|
||||
**Two Playwright MCP servers** (actively maintained, continuously updated):
|
||||
|
||||
- `playwright` - Browser automation (`npx @playwright/mcp@latest`)
|
||||
- `playwright-test` - Test runner with failure analysis (`npx playwright run-test-mcp-server`)
|
||||
|
||||
**Configuration example**:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"playwright": {
|
||||
"command": "npx",
|
||||
"args": ["@playwright/mcp@latest"]
|
||||
},
|
||||
"playwright-test": {
|
||||
"command": "npx",
|
||||
"args": ["playwright", "run-test-mcp-server"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Configuration:** Refer to your AI agent's documentation for MCP server setup instructions.
|
||||
|
||||
**Example (Enable):**
|
||||
```yaml
|
||||
tea_use_mcp_enhancements: true
|
||||
```
|
||||
|
||||
**Example (Disable):**
|
||||
```yaml
|
||||
tea_use_mcp_enhancements: false
|
||||
```
|
||||
|
||||
**Prerequisites:**
|
||||
1. MCP servers installed in IDE configuration
|
||||
2. `@playwright/mcp` package available globally or locally
|
||||
3. Browser binaries installed (`npx playwright install`)
|
||||
|
||||
**Related:**
|
||||
- [Enable MCP Enhancements Guide](/docs/how-to/customization/enable-tea-mcp-enhancements.md)
|
||||
- [TEA Overview - MCP Section](/docs/explanation/features/tea-overview.md#playwright-mcp-enhancements)
|
||||
- [Playwright MCP on npm](https://www.npmjs.com/package/@playwright/mcp)
|
||||
|
||||
---
|
||||
|
||||
## Core BMM Configuration (Inherited by TEA)
|
||||
|
||||
TEA also uses core BMM configuration options from `_bmad/bmm/config.yaml`:
|
||||
|
||||
### output_folder
|
||||
|
||||
**Type:** `string`
|
||||
|
||||
**Default:** `_bmad-output`
|
||||
|
||||
**Purpose:** Where TEA writes output files (test designs, reports, traceability matrices)
|
||||
|
||||
**Example:**
|
||||
```yaml
|
||||
output_folder: _bmad-output
|
||||
```
|
||||
|
||||
**TEA Output Files:**
|
||||
- `test-design-system.md` (from *test-design system-level)
|
||||
- `test-design-epic-N.md` (from *test-design epic-level)
|
||||
- `test-review.md` (from *test-review)
|
||||
- `traceability-matrix.md` (from *trace Phase 1)
|
||||
- `gate-decision-{gate_type}-{story_id}.md` (from *trace Phase 2)
|
||||
- `nfr-assessment.md` (from *nfr-assess)
|
||||
- `automation-summary.md` (from *automate)
|
||||
- `atdd-checklist-{story_id}.md` (from *atdd)
|
||||
|
||||
---
|
||||
|
||||
### user_skill_level
|
||||
|
||||
**Type:** `enum`
|
||||
|
||||
**Options:** `beginner` | `intermediate` | `expert`
|
||||
|
||||
**Default:** `intermediate`
|
||||
|
||||
**Purpose:** Affects how TEA explains concepts in chat responses
|
||||
|
||||
**Example:**
|
||||
```yaml
|
||||
user_skill_level: beginner
|
||||
```
|
||||
|
||||
**Impact on TEA:**
|
||||
- **Beginner:** More detailed explanations, links to concepts, verbose guidance
|
||||
- **Intermediate:** Balanced explanations, assumes basic knowledge
|
||||
- **Expert:** Concise, technical, minimal hand-holding
|
||||
|
||||
---
|
||||
|
||||
### project_name
|
||||
|
||||
**Type:** `string`
|
||||
|
||||
**Default:** Directory name
|
||||
|
||||
**Purpose:** Used in TEA-generated documentation and reports
|
||||
|
||||
**Example:**
|
||||
```yaml
|
||||
project_name: my-awesome-app
|
||||
```
|
||||
|
||||
**Used in:**
|
||||
- Report headers
|
||||
- Documentation titles
|
||||
- CI configuration comments
|
||||
|
||||
---
|
||||
|
||||
### communication_language
|
||||
|
||||
**Type:** `string`
|
||||
|
||||
**Default:** `english`
|
||||
|
||||
**Purpose:** Language for TEA chat responses
|
||||
|
||||
**Example:**
|
||||
```yaml
|
||||
communication_language: english
|
||||
```
|
||||
|
||||
**Supported:** Any language (TEA responds in specified language)
|
||||
|
||||
---
|
||||
|
||||
### document_output_language
|
||||
|
||||
**Type:** `string`
|
||||
|
||||
**Default:** `english`
|
||||
|
||||
**Purpose:** Language for TEA-generated documents (test designs, reports)
|
||||
|
||||
**Example:**
|
||||
```yaml
|
||||
document_output_language: english
|
||||
```
|
||||
|
||||
**Note:** Can differ from `communication_language` - chat in Spanish, generate docs in English.
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
TEA workflows may use environment variables for test configuration.
|
||||
|
||||
### Test Framework Variables
|
||||
|
||||
**Playwright:**
|
||||
```bash
|
||||
# .env
|
||||
BASE_URL=https://todomvc.com/examples/react/dist/
|
||||
API_BASE_URL=https://api.example.com
|
||||
TEST_USER_EMAIL=test@example.com
|
||||
TEST_USER_PASSWORD=password123
|
||||
```
|
||||
|
||||
**Cypress:**
|
||||
```bash
|
||||
# cypress.env.json or .env
|
||||
CYPRESS_BASE_URL=https://example.com
|
||||
CYPRESS_API_URL=https://api.example.com
|
||||
```
|
||||
|
||||
### CI/CD Variables
|
||||
|
||||
Set in CI platform (GitHub Actions secrets, GitLab CI variables):
|
||||
|
||||
```yaml
|
||||
# .github/workflows/test.yml
|
||||
env:
|
||||
BASE_URL: ${{ secrets.STAGING_URL }}
|
||||
API_KEY: ${{ secrets.API_KEY }}
|
||||
TEST_USER_EMAIL: ${{ secrets.TEST_USER }}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration Patterns
|
||||
|
||||
### Development vs Production
|
||||
|
||||
**Separate configs for environments:**
|
||||
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml
|
||||
output_folder: _bmad-output
|
||||
|
||||
# .env.development
|
||||
BASE_URL=http://localhost:3000
|
||||
API_BASE_URL=http://localhost:4000
|
||||
|
||||
# .env.staging
|
||||
BASE_URL=https://staging.example.com
|
||||
API_BASE_URL=https://api-staging.example.com
|
||||
|
||||
# .env.production (read-only tests only!)
|
||||
BASE_URL=https://example.com
|
||||
API_BASE_URL=https://api.example.com
|
||||
```
|
||||
|
||||
### Team vs Individual
|
||||
|
||||
**Team config (committed):**
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml.example (committed to repo)
|
||||
project_name: team-project
|
||||
output_folder: _bmad-output
|
||||
tea_use_playwright_utils: true
|
||||
tea_use_mcp_enhancements: false
|
||||
```
|
||||
|
||||
**Individual config (typically gitignored):**
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml (user adds to .gitignore)
|
||||
user_name: John Doe
|
||||
user_skill_level: expert
|
||||
tea_use_mcp_enhancements: true # Individual preference
|
||||
```
|
||||
|
||||
### Monorepo Configuration
|
||||
|
||||
**Root config:**
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml (root)
|
||||
project_name: monorepo-parent
|
||||
output_folder: _bmad-output
|
||||
```
|
||||
|
||||
**Package-specific:**
|
||||
```yaml
|
||||
# packages/web-app/_bmad/bmm/config.yaml
|
||||
project_name: web-app
|
||||
output_folder: ../../_bmad-output/web-app
|
||||
tea_use_playwright_utils: true
|
||||
|
||||
# packages/mobile-app/_bmad/bmm/config.yaml
|
||||
project_name: mobile-app
|
||||
output_folder: ../../_bmad-output/mobile-app
|
||||
tea_use_playwright_utils: false
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration Best Practices
|
||||
|
||||
### 1. Use Version Control Wisely
|
||||
|
||||
**Commit:**
|
||||
```
|
||||
_bmad/bmm/config.yaml.example # Template for team
|
||||
.nvmrc # Node version
|
||||
package.json # Dependencies
|
||||
```
|
||||
|
||||
**Recommended for .gitignore:**
|
||||
```
|
||||
_bmad/bmm/config.yaml # User-specific values
|
||||
.env # Secrets
|
||||
.env.local # Local overrides
|
||||
```
|
||||
|
||||
### 2. Document Required Setup
|
||||
|
||||
**In your README:**
|
||||
```markdown
|
||||
## Setup
|
||||
|
||||
1. Install BMad
|
||||
|
||||
2. Copy config template:
|
||||
cp _bmad/bmm/config.yaml.example _bmad/bmm/config.yaml
|
||||
|
||||
3. Edit config with your values:
|
||||
- Set user_name
|
||||
- Enable tea_use_playwright_utils if using playwright-utils
|
||||
- Enable tea_use_mcp_enhancements if MCPs configured
|
||||
```
|
||||
|
||||
### 3. Validate Configuration
|
||||
|
||||
**Check config is valid:**
|
||||
```bash
|
||||
# Check TEA config is set
|
||||
cat _bmad/bmm/config.yaml | grep tea_use
|
||||
|
||||
# Verify playwright-utils installed (if enabled)
|
||||
npm list @seontechnologies/playwright-utils
|
||||
|
||||
# Verify MCP servers configured (if enabled)
|
||||
# Check your IDE's MCP settings
|
||||
```
|
||||
|
||||
### 4. Keep Config Minimal
|
||||
|
||||
**Don't over-configure:**
|
||||
```yaml
|
||||
# ❌ Bad - overriding everything unnecessarily
|
||||
project_name: my-project
|
||||
user_name: John Doe
|
||||
user_skill_level: expert
|
||||
output_folder: custom/path
|
||||
planning_artifacts: custom/planning
|
||||
implementation_artifacts: custom/implementation
|
||||
project_knowledge: custom/docs
|
||||
tea_use_playwright_utils: true
|
||||
tea_use_mcp_enhancements: true
|
||||
communication_language: english
|
||||
document_output_language: english
|
||||
# Overriding 11 config options when most can use defaults
|
||||
|
||||
# ✅ Good - only essential overrides
|
||||
tea_use_playwright_utils: true
|
||||
output_folder: docs/testing
|
||||
# Only override what differs from defaults
|
||||
```
|
||||
|
||||
**Use defaults when possible** - only override what you actually need to change.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Configuration Not Loaded
|
||||
|
||||
**Problem:** TEA doesn't use my config values.
|
||||
|
||||
**Causes:**
|
||||
1. Config file in wrong location
|
||||
2. YAML syntax error
|
||||
3. Typo in config key
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check file exists
|
||||
ls -la _bmad/bmm/config.yaml
|
||||
|
||||
# Validate YAML syntax
|
||||
npm install -g js-yaml
|
||||
js-yaml _bmad/bmm/config.yaml
|
||||
|
||||
# Check for typos (compare to module.yaml)
|
||||
diff _bmad/bmm/config.yaml src/modules/bmm/module.yaml
|
||||
```
|
||||
|
||||
### Playwright Utils Not Working
|
||||
|
||||
**Problem:** `tea_use_playwright_utils: true` but TEA doesn't use utilities.
|
||||
|
||||
**Causes:**
|
||||
1. Package not installed
|
||||
2. Config file not saved
|
||||
3. Workflow run before config update
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Verify package installed
|
||||
npm list @seontechnologies/playwright-utils
|
||||
|
||||
# Check config value
|
||||
grep tea_use_playwright_utils _bmad/bmm/config.yaml
|
||||
|
||||
# Re-run workflow in fresh chat
|
||||
# (TEA loads config at workflow start)
|
||||
```
|
||||
|
||||
### MCP Enhancements Not Working
|
||||
|
||||
**Problem:** `tea_use_mcp_enhancements: true` but no browser opens.
|
||||
|
||||
**Causes:**
|
||||
1. MCP servers not configured in IDE
|
||||
2. MCP package not installed
|
||||
3. Browser binaries missing
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check MCP package available
|
||||
npx @playwright/mcp@latest --version
|
||||
|
||||
# Install browsers
|
||||
npx playwright install
|
||||
|
||||
# Verify IDE MCP config
|
||||
# Check ~/.cursor/config.json or VS Code settings
|
||||
```
|
||||
|
||||
### Config Changes Not Applied
|
||||
|
||||
**Problem:** Updated config but TEA still uses old values.
|
||||
|
||||
**Cause:** TEA loads config at workflow start.
|
||||
|
||||
**Solution:**
|
||||
1. Save `_bmad/bmm/config.yaml`
|
||||
2. Start fresh chat
|
||||
3. Run TEA workflow
|
||||
4. Config will be reloaded
|
||||
|
||||
**TEA doesn't reload config mid-chat** - always start fresh chat after config changes.
|
||||
|
||||
---
|
||||
|
||||
## Configuration Examples
|
||||
|
||||
### Recommended Setup (Full Stack)
|
||||
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml
|
||||
project_name: my-project
|
||||
user_skill_level: beginner # or intermediate/expert
|
||||
output_folder: _bmad-output
|
||||
tea_use_playwright_utils: true # Recommended
|
||||
tea_use_mcp_enhancements: true # Recommended
|
||||
```
|
||||
|
||||
**Why recommended:**
|
||||
- Playwright Utils: Production-ready fixtures and utilities
|
||||
- MCP enhancements: Live browser verification, visual debugging
|
||||
- Together: The three-part stack (see [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md))
|
||||
|
||||
**Prerequisites:**
|
||||
```bash
|
||||
npm install -D @seontechnologies/playwright-utils
|
||||
# Configure MCP servers in IDE (see Enable MCP Enhancements guide)
|
||||
```
|
||||
|
||||
**Best for:** Everyone (beginners learn good patterns from day one)
|
||||
|
||||
---
|
||||
|
||||
### Minimal Setup (Learning Only)
|
||||
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml
|
||||
project_name: my-project
|
||||
output_folder: _bmad-output
|
||||
tea_use_playwright_utils: false
|
||||
tea_use_mcp_enhancements: false
|
||||
```
|
||||
|
||||
**Best for:**
|
||||
- First-time TEA users (keep it simple initially)
|
||||
- Quick experiments
|
||||
- Learning basics before adding integrations
|
||||
|
||||
**Note:** Can enable integrations later as you learn
|
||||
|
||||
---
|
||||
|
||||
### Monorepo Setup
|
||||
|
||||
**Root config:**
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml (root)
|
||||
project_name: monorepo
|
||||
output_folder: _bmad-output
|
||||
tea_use_playwright_utils: true
|
||||
```
|
||||
|
||||
**Package configs:**
|
||||
```yaml
|
||||
# apps/web/_bmad/bmm/config.yaml
|
||||
project_name: web-app
|
||||
output_folder: ../../_bmad-output/web
|
||||
|
||||
# apps/api/_bmad/bmm/config.yaml
|
||||
project_name: api-service
|
||||
output_folder: ../../_bmad-output/api
|
||||
tea_use_playwright_utils: false # Using vanilla Playwright only
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Team Template
|
||||
|
||||
**Commit this template:**
|
||||
```yaml
|
||||
# _bmad/bmm/config.yaml.example
|
||||
# Copy to config.yaml and fill in your values
|
||||
|
||||
project_name: your-project-name
|
||||
user_name: Your Name
|
||||
user_skill_level: intermediate # beginner | intermediate | expert
|
||||
output_folder: _bmad-output
|
||||
planning_artifacts: _bmad-output/planning-artifacts
|
||||
implementation_artifacts: _bmad-output/implementation-artifacts
|
||||
project_knowledge: docs
|
||||
|
||||
# TEA Configuration (Recommended: Enable both for full stack)
|
||||
tea_use_playwright_utils: true # Recommended - production-ready utilities
|
||||
tea_use_mcp_enhancements: true # Recommended - live browser verification
|
||||
|
||||
# Languages
|
||||
communication_language: english
|
||||
document_output_language: english
|
||||
```
|
||||
|
||||
**Team instructions:**
|
||||
```markdown
|
||||
## Setup for New Team Members
|
||||
|
||||
1. Clone repo
|
||||
2. Copy config template:
|
||||
cp _bmad/bmm/config.yaml.example _bmad/bmm/config.yaml
|
||||
3. Edit with your name and preferences
|
||||
4. Install dependencies:
|
||||
npm install
|
||||
5. (Optional) Enable playwright-utils:
|
||||
npm install -D @seontechnologies/playwright-utils
|
||||
Set tea_use_playwright_utils: true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
### How-To Guides
|
||||
- [Set Up Test Framework](/docs/how-to/workflows/setup-test-framework.md)
|
||||
- [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md)
|
||||
- [Enable MCP Enhancements](/docs/how-to/customization/enable-tea-mcp-enhancements.md)
|
||||
|
||||
### Reference
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md)
|
||||
- [Knowledge Base Index](/docs/reference/tea/knowledge-base.md)
|
||||
- [Glossary](/docs/reference/glossary/index.md)
|
||||
|
||||
### Explanation
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md)
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md)
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -0,0 +1,340 @@
|
|||
---
|
||||
title: "TEA Knowledge Base Index"
|
||||
description: Complete index of TEA's 33 knowledge fragments for context engineering
|
||||
---
|
||||
|
||||
# TEA Knowledge Base Index
|
||||
|
||||
TEA uses 33 specialized knowledge fragments for context engineering. These fragments are loaded dynamically based on workflow needs via the `tea-index.csv` manifest.
|
||||
|
||||
## What is Context Engineering?
|
||||
|
||||
**Context engineering** is the practice of loading domain-specific standards into AI context automatically rather than relying on prompts alone.
|
||||
|
||||
Instead of asking AI to "write good tests" every time, TEA:
|
||||
1. Reads `tea-index.csv` to identify relevant fragments for the workflow
|
||||
2. Loads only the fragments needed (keeps context focused)
|
||||
3. Operates with domain-specific standards, not generic knowledge
|
||||
4. Produces consistent, production-ready tests across projects
|
||||
|
||||
**Example:**
|
||||
```
|
||||
User runs: *test-design
|
||||
|
||||
TEA reads tea-index.csv:
|
||||
- Loads: test-quality.md, test-priorities-matrix.md, risk-governance.md
|
||||
- Skips: network-recorder.md, burn-in.md (not needed for test design)
|
||||
|
||||
Result: Focused context, consistent quality standards
|
||||
```
|
||||
|
||||
## How Knowledge Loading Works
|
||||
|
||||
### 1. Workflow Trigger
|
||||
User runs a TEA workflow (e.g., `*test-design`)
|
||||
|
||||
### 2. Manifest Lookup
|
||||
TEA reads `src/modules/bmm/testarch/tea-index.csv`:
|
||||
```csv
|
||||
id,name,description,tags,fragment_file
|
||||
test-quality,Test Quality,Execution limits and isolation rules,quality;standards,knowledge/test-quality.md
|
||||
risk-governance,Risk Governance,Risk scoring and gate decisions,risk;governance,knowledge/risk-governance.md
|
||||
```
|
||||
|
||||
### 3. Dynamic Loading
|
||||
Only fragments needed for the workflow are loaded into context
|
||||
|
||||
### 4. Consistent Output
|
||||
AI operates with established patterns, producing consistent results
|
||||
|
||||
## Fragment Categories
|
||||
|
||||
### Architecture & Fixtures
|
||||
|
||||
Core patterns for test infrastructure and fixture composition.
|
||||
|
||||
| Fragment | Description | Key Topics |
|
||||
|----------|-------------|-----------|
|
||||
| [fixture-architecture](../../../src/modules/bmm/testarch/knowledge/fixture-architecture.md) | Pure function → Fixture → mergeTests composition with auto-cleanup | Testability, composition, reusability |
|
||||
| [network-first](../../../src/modules/bmm/testarch/knowledge/network-first.md) | Intercept-before-navigate workflow, HAR capture, deterministic waits | Flakiness prevention, network patterns |
|
||||
| [playwright-config](../../../src/modules/bmm/testarch/knowledge/playwright-config.md) | Environment switching, timeout standards, artifact outputs | Configuration, environments, CI |
|
||||
| [fixtures-composition](../../../src/modules/bmm/testarch/knowledge/fixtures-composition.md) | mergeTests composition patterns for combining utilities | Fixture merging, utility composition |
|
||||
|
||||
**Used in:** `*framework`, `*test-design`, `*atdd`, `*automate`, `*test-review`
|
||||
|
||||
---
|
||||
|
||||
### Data & Setup
|
||||
|
||||
Patterns for test data generation, authentication, and setup.
|
||||
|
||||
| Fragment | Description | Key Topics |
|
||||
|----------|-------------|-----------|
|
||||
| [data-factories](../../../src/modules/bmm/testarch/knowledge/data-factories.md) | Factory patterns with faker, overrides, API seeding, cleanup | Test data, factories, cleanup |
|
||||
| [email-auth](../../../src/modules/bmm/testarch/knowledge/email-auth.md) | Magic link extraction, state preservation, negative flows | Authentication, email testing |
|
||||
| [auth-session](../../../src/modules/bmm/testarch/knowledge/auth-session.md) | Token persistence, multi-user, API/browser authentication | Auth patterns, session management |
|
||||
|
||||
**Used in:** `*framework`, `*atdd`, `*automate`, `*test-review`
|
||||
|
||||
---
|
||||
|
||||
### Network & Reliability
|
||||
|
||||
Network interception, error handling, and reliability patterns.
|
||||
|
||||
| Fragment | Description | Key Topics |
|
||||
|----------|-------------|-----------|
|
||||
| [network-recorder](../../../src/modules/bmm/testarch/knowledge/network-recorder.md) | HAR record/playback, CRUD detection for offline testing | Offline testing, network replay |
|
||||
| [intercept-network-call](../../../src/modules/bmm/testarch/knowledge/intercept-network-call.md) | Network spy/stub, JSON parsing for UI tests | Mocking, interception, stubbing |
|
||||
| [error-handling](../../../src/modules/bmm/testarch/knowledge/error-handling.md) | Scoped exception handling, retry validation, telemetry logging | Error patterns, resilience |
|
||||
| [network-error-monitor](../../../src/modules/bmm/testarch/knowledge/network-error-monitor.md) | HTTP 4xx/5xx detection for UI tests | Error detection, monitoring |
|
||||
|
||||
**Used in:** `*atdd`, `*automate`, `*test-review`
|
||||
|
||||
---
|
||||
|
||||
### Test Execution & CI
|
||||
|
||||
CI/CD patterns, burn-in testing, and selective test execution.
|
||||
|
||||
| Fragment | Description | Key Topics |
|
||||
|----------|-------------|-----------|
|
||||
| [ci-burn-in](../../../src/modules/bmm/testarch/knowledge/ci-burn-in.md) | Staged jobs, shard orchestration, burn-in loops | CI/CD, flakiness detection |
|
||||
| [burn-in](../../../src/modules/bmm/testarch/knowledge/burn-in.md) | Smart test selection, git diff for CI optimization | Test selection, performance |
|
||||
| [selective-testing](../../../src/modules/bmm/testarch/knowledge/selective-testing.md) | Tag/grep usage, spec filters, diff-based runs | Test filtering, optimization |
|
||||
|
||||
**Used in:** `*ci`, `*test-review`
|
||||
|
||||
---
|
||||
|
||||
### Quality & Standards
|
||||
|
||||
Test quality standards, test level selection, and TDD patterns.
|
||||
|
||||
| Fragment | Description | Key Topics |
|
||||
|----------|-------------|-----------|
|
||||
| [test-quality](../../../src/modules/bmm/testarch/knowledge/test-quality.md) | Execution limits, isolation rules, green criteria | DoD, best practices, anti-patterns |
|
||||
| [test-levels-framework](../../../src/modules/bmm/testarch/knowledge/test-levels-framework.md) | Guidelines for unit, integration, E2E selection | Test pyramid, level selection |
|
||||
| [test-priorities-matrix](../../../src/modules/bmm/testarch/knowledge/test-priorities-matrix.md) | P0-P3 criteria, coverage targets, execution ordering | Prioritization, risk-based testing |
|
||||
| [test-healing-patterns](../../../src/modules/bmm/testarch/knowledge/test-healing-patterns.md) | Common failure patterns and automated fixes | Debugging, healing, fixes |
|
||||
| [component-tdd](../../../src/modules/bmm/testarch/knowledge/component-tdd.md) | Red→green→refactor workflow, provider isolation | TDD, component testing |
|
||||
|
||||
**Used in:** `*test-design`, `*atdd`, `*automate`, `*test-review`, `*trace`
|
||||
|
||||
---
|
||||
|
||||
### Risk & Gates
|
||||
|
||||
Risk assessment, governance, and gate decision frameworks.
|
||||
|
||||
| Fragment | Description | Key Topics |
|
||||
|----------|-------------|-----------|
|
||||
| [risk-governance](../../../src/modules/bmm/testarch/knowledge/risk-governance.md) | Scoring matrix, category ownership, gate decision rules | Risk assessment, governance |
|
||||
| [probability-impact](../../../src/modules/bmm/testarch/knowledge/probability-impact.md) | Probability × impact scale for scoring matrix | Risk scoring, impact analysis |
|
||||
| [nfr-criteria](../../../src/modules/bmm/testarch/knowledge/nfr-criteria.md) | Security, performance, reliability, maintainability status | NFRs, compliance, enterprise |
|
||||
|
||||
**Used in:** `*test-design`, `*nfr-assess`, `*trace`
|
||||
|
||||
---
|
||||
|
||||
### Selectors & Timing
|
||||
|
||||
Selector resilience, race condition debugging, and visual debugging.
|
||||
|
||||
| Fragment | Description | Key Topics |
|
||||
|----------|-------------|-----------|
|
||||
| [selector-resilience](../../../src/modules/bmm/testarch/knowledge/selector-resilience.md) | Robust selector strategies and debugging | Selectors, locators, resilience |
|
||||
| [timing-debugging](../../../src/modules/bmm/testarch/knowledge/timing-debugging.md) | Race condition identification and deterministic fixes | Race conditions, timing issues |
|
||||
| [visual-debugging](../../../src/modules/bmm/testarch/knowledge/visual-debugging.md) | Trace viewer usage, artifact expectations | Debugging, trace viewer, artifacts |
|
||||
|
||||
**Used in:** `*atdd`, `*automate`, `*test-review`
|
||||
|
||||
---
|
||||
|
||||
### Feature Flags & Testing Patterns
|
||||
|
||||
Feature flag testing, contract testing, and API testing patterns.
|
||||
|
||||
| Fragment | Description | Key Topics |
|
||||
|----------|-------------|-----------|
|
||||
| [feature-flags](../../../src/modules/bmm/testarch/knowledge/feature-flags.md) | Enum management, targeting helpers, cleanup, checklists | Feature flags, toggles |
|
||||
| [contract-testing](../../../src/modules/bmm/testarch/knowledge/contract-testing.md) | Pact publishing, provider verification, resilience | Contract testing, Pact |
|
||||
| [api-testing-patterns](../../../src/modules/bmm/testarch/knowledge/api-testing-patterns.md) | Pure API patterns without browser | API testing, backend testing |
|
||||
|
||||
**Used in:** `*test-design`, `*atdd`, `*automate`
|
||||
|
||||
---
|
||||
|
||||
### Playwright-Utils Integration
|
||||
|
||||
Patterns for using `@seontechnologies/playwright-utils` package (9 utilities).
|
||||
|
||||
| Fragment | Description | Key Topics |
|
||||
|----------|-------------|-----------|
|
||||
| [api-request](../../../src/modules/bmm/testarch/knowledge/api-request.md) | Typed HTTP client, schema validation, retry logic | API calls, HTTP, validation |
|
||||
| [auth-session](../../../src/modules/bmm/testarch/knowledge/auth-session.md) | Token persistence, multi-user, API/browser authentication | Auth patterns, session management |
|
||||
| [network-recorder](../../../src/modules/bmm/testarch/knowledge/network-recorder.md) | HAR record/playback, CRUD detection for offline testing | Offline testing, network replay |
|
||||
| [intercept-network-call](../../../src/modules/bmm/testarch/knowledge/intercept-network-call.md) | Network spy/stub, JSON parsing for UI tests | Mocking, interception, stubbing |
|
||||
| [recurse](../../../src/modules/bmm/testarch/knowledge/recurse.md) | Async polling for API responses, background jobs | Polling, eventual consistency |
|
||||
| [log](../../../src/modules/bmm/testarch/knowledge/log.md) | Structured logging for API and UI tests | Logging, debugging, reporting |
|
||||
| [file-utils](../../../src/modules/bmm/testarch/knowledge/file-utils.md) | CSV/XLSX/PDF/ZIP handling with download support | File validation, exports |
|
||||
| [burn-in](../../../src/modules/bmm/testarch/knowledge/burn-in.md) | Smart test selection with git diff analysis | CI optimization, selective testing |
|
||||
| [network-error-monitor](../../../src/modules/bmm/testarch/knowledge/network-error-monitor.md) | Auto-detect HTTP 4xx/5xx errors during tests | Error monitoring, silent failures |
|
||||
|
||||
**Note:** `fixtures-composition` is listed under Architecture & Fixtures (general Playwright `mergeTests` pattern, applies to all fixtures).
|
||||
|
||||
**Used in:** `*framework` (if `tea_use_playwright_utils: true`), `*atdd`, `*automate`, `*test-review`, `*ci`
|
||||
|
||||
**Official Docs:** <https://seontechnologies.github.io/playwright-utils/>
|
||||
|
||||
---
|
||||
|
||||
## Fragment Manifest (tea-index.csv)
|
||||
|
||||
**Location:** `src/modules/bmm/testarch/tea-index.csv`
|
||||
|
||||
**Purpose:** Tracks all knowledge fragments and their usage in workflows
|
||||
|
||||
**Structure:**
|
||||
```csv
|
||||
id,name,description,tags,fragment_file
|
||||
test-quality,Test Quality,Execution limits and isolation rules,quality;standards,knowledge/test-quality.md
|
||||
risk-governance,Risk Governance,Risk scoring and gate decisions,risk;governance,knowledge/risk-governance.md
|
||||
```
|
||||
|
||||
**Columns:**
|
||||
- `id` - Unique fragment identifier (kebab-case)
|
||||
- `name` - Human-readable fragment name
|
||||
- `description` - What the fragment covers
|
||||
- `tags` - Searchable tags (semicolon-separated)
|
||||
- `fragment_file` - Relative path to fragment markdown file
|
||||
|
||||
**Fragment Location:** `src/modules/bmm/testarch/knowledge/` (all 33 fragments in single directory)
|
||||
|
||||
**Manifest:** `src/modules/bmm/testarch/tea-index.csv`
|
||||
|
||||
---
|
||||
|
||||
## Workflow Fragment Loading
|
||||
|
||||
Each TEA workflow loads specific fragments:
|
||||
|
||||
### *framework
|
||||
**Key Fragments:**
|
||||
- fixture-architecture.md
|
||||
- playwright-config.md
|
||||
- fixtures-composition.md
|
||||
|
||||
**Purpose:** Test infrastructure patterns and fixture composition
|
||||
|
||||
**Note:** Loads additional fragments based on framework choice (Playwright/Cypress) and config (`tea_use_playwright_utils`).
|
||||
|
||||
---
|
||||
|
||||
### *test-design
|
||||
**Key Fragments:**
|
||||
- test-quality.md
|
||||
- test-priorities-matrix.md
|
||||
- test-levels-framework.md
|
||||
- risk-governance.md
|
||||
- probability-impact.md
|
||||
|
||||
**Purpose:** Risk assessment and test planning standards
|
||||
|
||||
**Note:** Loads additional fragments based on mode (system-level vs epic-level) and focus areas.
|
||||
|
||||
---
|
||||
|
||||
### *atdd
|
||||
**Key Fragments:**
|
||||
- test-quality.md
|
||||
- component-tdd.md
|
||||
- fixture-architecture.md
|
||||
- network-first.md
|
||||
- data-factories.md
|
||||
- selector-resilience.md
|
||||
- timing-debugging.md
|
||||
- test-healing-patterns.md
|
||||
|
||||
**Purpose:** TDD patterns and test generation standards
|
||||
|
||||
**Note:** Loads auth, network, and utility fragments based on feature requirements.
|
||||
|
||||
---
|
||||
|
||||
### *automate
|
||||
**Key Fragments:**
|
||||
- test-quality.md
|
||||
- test-levels-framework.md
|
||||
- test-priorities-matrix.md
|
||||
- fixture-architecture.md
|
||||
- network-first.md
|
||||
- selector-resilience.md
|
||||
- test-healing-patterns.md
|
||||
- timing-debugging.md
|
||||
|
||||
**Purpose:** Comprehensive test generation with quality standards
|
||||
|
||||
**Note:** Loads additional fragments for data factories, auth, network utilities based on test needs.
|
||||
|
||||
---
|
||||
|
||||
### *test-review
|
||||
**Key Fragments:**
|
||||
- test-quality.md
|
||||
- test-healing-patterns.md
|
||||
- selector-resilience.md
|
||||
- timing-debugging.md
|
||||
- visual-debugging.md
|
||||
- network-first.md
|
||||
- test-levels-framework.md
|
||||
- fixture-architecture.md
|
||||
|
||||
**Purpose:** Comprehensive quality review against all standards
|
||||
|
||||
**Note:** Loads all applicable playwright-utils fragments when `tea_use_playwright_utils: true`.
|
||||
|
||||
---
|
||||
|
||||
### *ci
|
||||
**Key Fragments:**
|
||||
- ci-burn-in.md
|
||||
- burn-in.md
|
||||
- selective-testing.md
|
||||
- playwright-config.md
|
||||
|
||||
**Purpose:** CI/CD best practices and optimization
|
||||
|
||||
---
|
||||
|
||||
### *nfr-assess
|
||||
**Key Fragments:**
|
||||
- nfr-criteria.md
|
||||
- risk-governance.md
|
||||
- probability-impact.md
|
||||
|
||||
**Purpose:** NFR assessment frameworks and decision rules
|
||||
|
||||
---
|
||||
|
||||
### *trace
|
||||
**Key Fragments:**
|
||||
- test-priorities-matrix.md
|
||||
- risk-governance.md
|
||||
- test-quality.md
|
||||
|
||||
**Purpose:** Traceability and gate decision standards
|
||||
|
||||
**Note:** Loads nfr-criteria.md if NFR assessment is part of gate decision.
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - How knowledge base fits in TEA
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - Context engineering philosophy
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - Workflows that use fragments
|
||||
|
||||
---
|
||||
|
||||
Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)
|
||||
|
|
@ -1,260 +0,0 @@
|
|||
---
|
||||
title: "Getting Started with BMad Game Development"
|
||||
description: Build games with BMad's Game Development Module
|
||||
---
|
||||
|
||||
|
||||
Build games faster using AI-powered workflows with specialized game development agents that guide you through preproduction, design, architecture, and implementation.
|
||||
|
||||
:::note[Module Extension]
|
||||
BMGD (BMad Game Development) is a module that extends BMad Method. You'll need BMad installed first—see the [BMad v6 tutorial](/docs/tutorials/getting-started/getting-started-bmadv6.md) if you haven't installed it yet.
|
||||
:::
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- Install and configure the BMGD module
|
||||
- Understand game development phases and specialized agents
|
||||
- Create a Game Brief and Game Design Document (GDD)
|
||||
- Progress from concept to working game code
|
||||
|
||||
:::note[Prerequisites]
|
||||
- **BMad Method installed** — Follow the main installation guide first
|
||||
- **A game idea** — Even a rough concept is enough to start
|
||||
- **AI-powered IDE** — Claude Code, Cursor, Windsurf, or similar
|
||||
:::
|
||||
|
||||
:::tip[Quick Path]
|
||||
**Install** → `npx bmad-method install` (select BMGD module)
|
||||
**Preproduction** → Game Designer creates Game Brief
|
||||
**Design** → Game Designer creates GDD (and Narrative if story-driven)
|
||||
**Technical** → Game Architect creates Architecture
|
||||
**Production** → Game SM manages sprints, Game Dev implements
|
||||
**Always use fresh chats** for each workflow to avoid context issues.
|
||||
:::
|
||||
|
||||
## Understanding BMGD
|
||||
|
||||
BMGD follows four game development phases with specialized agents for each:
|
||||
|
||||
| Phase | Name | What Happens |
|
||||
| ----- | ------------- | ----------------------------------------------------------------- |
|
||||
| 1 | Preproduction | Capture game vision, create Game Brief *(optional brainstorming)* |
|
||||
| 2 | Design | Detail mechanics, systems, narrative in GDD |
|
||||
| 3 | Technical | Plan engine, architecture, technical decisions |
|
||||
| 4 | Production | Build game in sprints, story by story |
|
||||
|
||||

|
||||
|
||||
*Complete visual flowchart showing all phases, workflows, and agents for game development.*
|
||||
|
||||
### Game Development Agents
|
||||
|
||||
| Agent | When to Use |
|
||||
| --------------------- | ----------------------------------------- |
|
||||
| **Game Designer** | Brainstorming, Game Brief, GDD, Narrative |
|
||||
| **Game Architect** | Architecture, technical decisions |
|
||||
| **Game Developer** | Implementation, code reviews |
|
||||
| **Game Scrum Master** | Sprint planning, story management |
|
||||
| **Game QA** | Test framework, test design, automation |
|
||||
| **Game Solo Dev** | Quick prototyping, indie development |
|
||||
|
||||
## Installation
|
||||
|
||||
If you haven't installed BMad yet:
|
||||
|
||||
```bash
|
||||
npx bmad-method install
|
||||
```
|
||||
|
||||
Or add BMGD to an existing installation:
|
||||
|
||||
```bash
|
||||
npx bmad-method install --add-module bmgd
|
||||
```
|
||||
|
||||
Verify your installation:
|
||||
|
||||
```
|
||||
your-project/
|
||||
├── _bmad/
|
||||
│ ├── bmgd/ # Game development module
|
||||
│ │ ├── agents/ # Game-specific agents
|
||||
│ │ ├── workflows/ # Game-specific workflows
|
||||
│ │ └── config.yaml # Module config
|
||||
│ ├── bmm/ # Core method module
|
||||
│ └── core/ # Core utilities
|
||||
├── _bmad-output/ # Generated artifacts (created later)
|
||||
└── .claude/ # IDE configuration (if using Claude Code)
|
||||
```
|
||||
|
||||
## Step 1: Create Your Game Brief (Preproduction)
|
||||
|
||||
Load the **Game Designer** agent in your IDE, wait for the menu, then start with your game concept.
|
||||
|
||||
### Optional: Brainstorm First
|
||||
|
||||
If you have a vague idea and want help developing it:
|
||||
|
||||
```
|
||||
Run brainstorm-game
|
||||
```
|
||||
|
||||
The agent guides you through game-specific ideation techniques to refine your concept.
|
||||
|
||||
### Create the Game Brief
|
||||
|
||||
```
|
||||
Run create-game-brief
|
||||
```
|
||||
|
||||
The Game Designer walks you through:
|
||||
- **Game concept** — Core idea and unique selling points
|
||||
- **Design pillars** — The 3-5 principles that guide all decisions
|
||||
- **Target market** — Who plays this game?
|
||||
- **Fundamentals** — Platform, genre, scope, team size
|
||||
|
||||
When complete, you'll have `game-brief.md` in your `_bmad-output/` folder.
|
||||
|
||||
:::caution[Fresh Chats]
|
||||
Always start a fresh chat for each workflow. This prevents context limitations from causing issues.
|
||||
:::
|
||||
|
||||
## Step 2: Design Your Game
|
||||
|
||||
With your Game Brief complete, detail your game's design.
|
||||
|
||||
### Create the GDD
|
||||
|
||||
**Start a fresh chat** with the **Game Designer** agent.
|
||||
|
||||
```
|
||||
Run create-gdd
|
||||
```
|
||||
|
||||
The agent guides you through mechanics, systems, and game-type-specific sections. BMGD offers 24 game type templates that provide genre-specific structure.
|
||||
|
||||
When complete, you'll have `gdd.md` (or sharded into `gdd/` for large documents).
|
||||
|
||||
:::note[Narrative Design (Optional)]
|
||||
For story-driven games, start a fresh chat and run `narrative` to create a Narrative Design Document covering story, characters, world, and dialogue.
|
||||
:::
|
||||
|
||||
:::tip[Check Your Status]
|
||||
Unsure what's next? Load any agent and run `workflow-status`. It tells you the next recommended workflow.
|
||||
:::
|
||||
|
||||
## Step 3: Plan Your Architecture
|
||||
|
||||
**Start a fresh chat** with the **Game Architect** agent.
|
||||
|
||||
```
|
||||
Run create-architecture
|
||||
```
|
||||
|
||||
The architect guides you through:
|
||||
- **Engine selection** — Unity, Unreal, Godot, custom, etc.
|
||||
- **System design** — Core game systems and how they interact
|
||||
- **Technical patterns** — Architecture patterns suited to your game
|
||||
- **Structure** — Project organization and conventions
|
||||
|
||||
When complete, you'll have `game-architecture.md`.
|
||||
|
||||
## Step 4: Build Your Game
|
||||
|
||||
Once planning is complete, move to production. **Each workflow should run in a fresh chat.**
|
||||
|
||||
### Initialize Sprint Planning
|
||||
|
||||
Load the **Game Scrum Master** agent and run `sprint-planning`. This creates `sprint-status.yaml` to track all epics and stories.
|
||||
|
||||
### The Build Cycle
|
||||
|
||||
For each story, repeat this cycle with fresh chats:
|
||||
|
||||
| Step | Agent | Workflow | Purpose |
|
||||
| ---- | -------- | -------------- | ---------------------------------- |
|
||||
| 1 | Game SM | `create-story` | Create story file from epic |
|
||||
| 2 | Game Dev | `dev-story` | Implement the story |
|
||||
| 3 | Game QA | `automate` | Generate tests *(optional)* |
|
||||
| 4 | Game Dev | `code-review` | Quality validation *(recommended)* |
|
||||
|
||||
After completing all stories in an epic, load the **Game SM** and run `retrospective`.
|
||||
|
||||
### Quick Prototyping Alternative
|
||||
|
||||
For rapid iteration or indie development, load the **Game Solo Dev** agent:
|
||||
- `quick-prototype` — Rapid prototyping
|
||||
- `quick-dev` — Flexible development without full sprint structure
|
||||
|
||||
## What You've Accomplished
|
||||
|
||||
You've learned the foundation of building games with BMad:
|
||||
|
||||
- Installed the BMGD module
|
||||
- Created a Game Brief capturing your vision
|
||||
- Detailed your design in a GDD
|
||||
- Planned your technical architecture
|
||||
- Understood the build cycle for implementation
|
||||
|
||||
Your project now has:
|
||||
|
||||
```
|
||||
your-project/
|
||||
├── _bmad/ # BMad configuration
|
||||
├── _bmad-output/
|
||||
│ ├── game-brief.md # Your game vision
|
||||
│ ├── gdd.md # Game Design Document
|
||||
│ ├── narrative-design.md # Story design (if applicable)
|
||||
│ ├── game-architecture.md # Technical decisions
|
||||
│ ├── epics/ # Epic and story files
|
||||
│ └── sprint-status.yaml # Sprint tracking
|
||||
└── ...
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Command | Agent | Purpose |
|
||||
| ---------------------- | -------------- | ----------------------------- |
|
||||
| `*brainstorm-game` | Game Designer | Guided game ideation |
|
||||
| `*create-game-brief` | Game Designer | Create Game Brief |
|
||||
| `*create-gdd` | Game Designer | Create Game Design Document |
|
||||
| `*narrative` | Game Designer | Create Narrative Design |
|
||||
| `*create-architecture` | Game Architect | Create game architecture |
|
||||
| `*sprint-planning` | Game SM | Initialize sprint tracking |
|
||||
| `*create-story` | Game SM | Create a story file |
|
||||
| `*dev-story` | Game Dev | Implement a story |
|
||||
| `*code-review` | Game Dev | Review implemented code |
|
||||
| `*workflow-status` | Any | Check progress and next steps |
|
||||
|
||||
## Common Questions
|
||||
|
||||
**Do I need to create all documents?**
|
||||
At minimum, create a Game Brief and GDD. Architecture is highly recommended. Narrative Design is only needed for story-driven games.
|
||||
|
||||
**Can I use the Game Solo Dev for everything?**
|
||||
Yes, for smaller projects or rapid prototyping. For larger games, the specialized agents provide more thorough guidance.
|
||||
|
||||
**What game types are supported?**
|
||||
BMGD includes 24 game type templates (RPG, platformer, puzzle, strategy, etc.) that provide genre-specific GDD sections.
|
||||
|
||||
**Can I change my design later?**
|
||||
Yes. Documents are living artifacts—return to update them as your vision evolves. The SM agent has `correct-course` for scope changes.
|
||||
|
||||
## Getting Help
|
||||
|
||||
- **During workflows** — Agents guide you with questions and explanations
|
||||
- **Community** — [Discord](https://discord.gg/gk8jAdXWmj) (#bmad-method-help, #report-bugs-and-issues)
|
||||
- **Documentation** — [BMGD Workflow Reference](/docs/reference/workflows/bmgd-workflows.md)
|
||||
- **Video tutorials** — [BMad Code YouTube](https://www.youtube.com/@BMadCode)
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
:::tip[Remember These]
|
||||
- **Always use fresh chats** — Load agents in new chats for each workflow
|
||||
- **Game Brief first** — It informs everything that follows
|
||||
- **Use game type templates** — 24 templates provide genre-specific GDD structure
|
||||
- **Documents evolve** — Return to update them as your vision grows
|
||||
- **Solo Dev for speed** — Use Game Solo Dev for rapid prototyping
|
||||
:::
|
||||
|
||||
Ready to start? Load the **Game Designer** agent and run `create-game-brief` to capture your game vision.
|
||||
|
|
@ -0,0 +1,444 @@
|
|||
---
|
||||
title: "Getting Started with Test Architect"
|
||||
description: Learn Test Architect fundamentals by generating and running tests for an existing demo app in 30 minutes
|
||||
---
|
||||
|
||||
Welcome! **Test Architect (TEA) Lite** is the simplest way to get started with TEA - just use `*automate` to generate tests for existing features. Perfect for beginners who want to learn TEA fundamentals quickly.
|
||||
|
||||
## What You'll Build
|
||||
|
||||
By the end of this 30-minute tutorial, you'll have:
|
||||
- A working Playwright test framework
|
||||
- Your first risk-based test plan
|
||||
- Passing tests for an existing demo app feature
|
||||
|
||||
:::note[Prerequisites]
|
||||
- Node.js installed (v20 or later)
|
||||
- 30 minutes of focused time
|
||||
- We'll use TodoMVC (<https://todomvc.com/examples/react/>) as our demo app
|
||||
:::
|
||||
|
||||
:::tip[Quick Path]
|
||||
Load TEA (`*tea`) → scaffold framework (`*framework`) → create test plan (`*test-design`) → generate tests (`*automate`) → run with `npx playwright test`.
|
||||
:::
|
||||
|
||||
## TEA Approaches Explained
|
||||
|
||||
Before we start, understand the three ways to use TEA:
|
||||
|
||||
- **TEA Lite** (this tutorial): Beginner using just `*automate` to test existing features
|
||||
- **TEA Solo**: Using TEA standalone without full BMad Method integration
|
||||
- **TEA Integrated**: Full BMad Method with all TEA workflows across phases
|
||||
|
||||
This tutorial focuses on **TEA Lite** - the fastest way to see TEA in action.
|
||||
|
||||
## Step 0: Setup (2 minutes)
|
||||
|
||||
We'll test TodoMVC, a standard demo app used across testing documentation.
|
||||
|
||||
**Demo App:** <https://todomvc.com/examples/react/dist/>
|
||||
|
||||
No installation needed - TodoMVC runs in your browser. Open the link above and:
|
||||
1. Add a few todos (type and press Enter)
|
||||
2. Mark some as complete (click checkbox)
|
||||
3. Try the "All", "Active", "Completed" filters
|
||||
|
||||
You've just explored the features we'll test!
|
||||
|
||||
## Step 1: Install BMad and Scaffold Framework (10 minutes)
|
||||
|
||||
### Install BMad Method
|
||||
|
||||
Install BMad (see installation guide for latest command).
|
||||
|
||||
When prompted:
|
||||
- **Select modules:** Choose "BMM: BMad Method" (press Space, then Enter)
|
||||
- **Project name:** Keep default or enter your project name
|
||||
- **Experience level:** Choose "beginner" for this tutorial
|
||||
- **Planning artifacts folder:** Keep default
|
||||
- **Implementation artifacts folder:** Keep default
|
||||
- **Project knowledge folder:** Keep default
|
||||
- **Enable TEA Playwright Model Context Protocol (MCP) enhancements?** Choose "No" for now (we'll explore this later)
|
||||
- **Using playwright-utils?** Choose "No" for now (we'll explore this later)
|
||||
|
||||
BMad is now installed! You'll see a `_bmad/` folder in your project.
|
||||
|
||||
### Load TEA Agent
|
||||
|
||||
Start a new chat with your AI assistant (Claude, etc.) and type:
|
||||
|
||||
```
|
||||
*tea
|
||||
```
|
||||
|
||||
This loads the Test Architect agent. You'll see TEA's menu with available workflows.
|
||||
|
||||
### Scaffold Test Framework
|
||||
|
||||
In your chat, run:
|
||||
|
||||
```
|
||||
*framework
|
||||
```
|
||||
|
||||
TEA will ask you questions:
|
||||
|
||||
**Q: What's your tech stack?**
|
||||
A: "We're testing a React web application (TodoMVC)"
|
||||
|
||||
**Q: Which test framework?**
|
||||
A: "Playwright"
|
||||
|
||||
**Q: Testing scope?**
|
||||
A: "End-to-end (E2E) testing for a web application"
|
||||
|
||||
**Q: Continuous integration/continuous deployment (CI/CD) platform?**
|
||||
A: "GitHub Actions" (or your preference)
|
||||
|
||||
TEA will generate:
|
||||
- `tests/` directory with Playwright config
|
||||
- `playwright.config.ts` with base configuration
|
||||
- Sample test structure
|
||||
- `.env.example` for environment variables
|
||||
- `.nvmrc` for Node version
|
||||
|
||||
**Verify the setup:**
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npx playwright install
|
||||
```
|
||||
|
||||
You now have a production-ready test framework!
|
||||
|
||||
## Step 2: Your First Test Design (5 minutes)
|
||||
|
||||
Test design is where TEA shines - risk-based planning before writing tests.
|
||||
|
||||
### Run Test Design
|
||||
|
||||
In your chat with TEA, run:
|
||||
|
||||
```
|
||||
*test-design
|
||||
```
|
||||
|
||||
**Q: System-level or epic-level?**
|
||||
A: "Epic-level - I want to test TodoMVC's basic functionality"
|
||||
|
||||
**Q: What feature are you testing?**
|
||||
A: "TodoMVC's core operations - creating, completing, and deleting todos"
|
||||
|
||||
**Q: Any specific risks or concerns?**
|
||||
A: "We want to ensure the filter buttons (All, Active, Completed) work correctly"
|
||||
|
||||
TEA will analyze and create `test-design-epic-1.md` with:
|
||||
|
||||
1. **Risk Assessment**
|
||||
- Probability × Impact scoring
|
||||
- Risk categories (TECH, SEC, PERF, DATA, BUS, OPS)
|
||||
- High-risk areas identified
|
||||
|
||||
2. **Test Priorities**
|
||||
- P0: Critical path (creating and displaying todos)
|
||||
- P1: High value (completing todos, filters)
|
||||
- P2: Medium value (deleting todos)
|
||||
- P3: Low value (edge cases)
|
||||
|
||||
3. **Coverage Strategy**
|
||||
- E2E tests for user workflows
|
||||
- Which scenarios need testing
|
||||
- Suggested test structure
|
||||
|
||||
**Review the test design file** - notice how TEA provides a systematic approach to what needs testing and why.
|
||||
|
||||
## Step 3: Generate Tests for Existing Features (5 minutes)
|
||||
|
||||
Now the magic happens - TEA generates tests based on your test design.
|
||||
|
||||
### Run Automate
|
||||
|
||||
In your chat with TEA, run:
|
||||
|
||||
```
|
||||
*automate
|
||||
```
|
||||
|
||||
**Q: What are you testing?**
|
||||
A: "TodoMVC React app at <https://todomvc.com/examples/react/dist/> - focus on the test design we just created"
|
||||
|
||||
**Q: Reference existing docs?**
|
||||
A: "Yes, use test-design-epic-1.md"
|
||||
|
||||
**Q: Any specific test scenarios?**
|
||||
A: "Cover the P0 and P1 scenarios from the test design"
|
||||
|
||||
TEA will generate:
|
||||
|
||||
**`tests/e2e/todomvc.spec.ts`** with tests like:
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test.describe('TodoMVC - Core Functionality', () => {
|
||||
test.beforeEach(async ({ page }) => {
|
||||
await page.goto('https://todomvc.com/examples/react/dist/');
|
||||
});
|
||||
|
||||
test('should create a new todo', async ({ page }) => {
|
||||
// TodoMVC uses a simple input without placeholder or test IDs
|
||||
const todoInput = page.locator('.new-todo');
|
||||
await todoInput.fill('Buy groceries');
|
||||
await todoInput.press('Enter');
|
||||
|
||||
// Verify todo appears in list
|
||||
await expect(page.locator('.todo-list li')).toContainText('Buy groceries');
|
||||
});
|
||||
|
||||
test('should mark todo as complete', async ({ page }) => {
|
||||
// Create a todo
|
||||
const todoInput = page.locator('.new-todo');
|
||||
await todoInput.fill('Complete tutorial');
|
||||
await todoInput.press('Enter');
|
||||
|
||||
// Mark as complete using the toggle checkbox
|
||||
await page.locator('.todo-list li .toggle').click();
|
||||
|
||||
// Verify completed state
|
||||
await expect(page.locator('.todo-list li')).toHaveClass(/completed/);
|
||||
});
|
||||
|
||||
test('should filter todos by status', async ({ page }) => {
|
||||
// Create multiple todos
|
||||
const todoInput = page.locator('.new-todo');
|
||||
await todoInput.fill('Buy groceries');
|
||||
await todoInput.press('Enter');
|
||||
await todoInput.fill('Write tests');
|
||||
await todoInput.press('Enter');
|
||||
|
||||
// Complete the first todo ("Buy groceries")
|
||||
await page.locator('.todo-list li .toggle').first().click();
|
||||
|
||||
// Test Active filter (shows only incomplete todos)
|
||||
await page.locator('.filters a[href="#/active"]').click();
|
||||
await expect(page.locator('.todo-list li')).toHaveCount(1);
|
||||
await expect(page.locator('.todo-list li')).toContainText('Write tests');
|
||||
|
||||
// Test Completed filter (shows only completed todos)
|
||||
await page.locator('.filters a[href="#/completed"]').click();
|
||||
await expect(page.locator('.todo-list li')).toHaveCount(1);
|
||||
await expect(page.locator('.todo-list li')).toContainText('Buy groceries');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
TEA also creates:
|
||||
- **`tests/README.md`** - How to run tests, project conventions
|
||||
- **Definition of Done summary** - What makes a test "good"
|
||||
|
||||
### With Playwright Utils (Optional Enhancement)
|
||||
|
||||
If you have `tea_use_playwright_utils: true` in your config, TEA generates tests using production-ready utilities:
|
||||
|
||||
**Vanilla Playwright:**
|
||||
```typescript
|
||||
test('should mark todo as complete', async ({ page, request }) => {
|
||||
// Manual API call
|
||||
const response = await request.post('/api/todos', {
|
||||
data: { title: 'Complete tutorial' }
|
||||
});
|
||||
const todo = await response.json();
|
||||
|
||||
await page.goto('/');
|
||||
await page.locator(`.todo-list li:has-text("${todo.title}") .toggle`).click();
|
||||
await expect(page.locator('.todo-list li')).toHaveClass(/completed/);
|
||||
});
|
||||
```
|
||||
|
||||
**With Playwright Utils:**
|
||||
```typescript
|
||||
import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
|
||||
import { expect } from '@playwright/test';
|
||||
|
||||
test('should mark todo as complete', async ({ page, apiRequest }) => {
|
||||
// Typed API call with cleaner syntax
|
||||
const { status, body: todo } = await apiRequest({
|
||||
method: 'POST',
|
||||
path: '/api/todos',
|
||||
body: { title: 'Complete tutorial' }
|
||||
});
|
||||
|
||||
expect(status).toBe(201);
|
||||
await page.goto('/');
|
||||
await page.locator(`.todo-list li:has-text("${todo.title}") .toggle`).click();
|
||||
await expect(page.locator('.todo-list li')).toHaveClass(/completed/);
|
||||
});
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Type-safe API responses (`{ status, body }`)
|
||||
- Automatic retry for 5xx errors
|
||||
- Built-in schema validation
|
||||
- Cleaner, more maintainable code
|
||||
|
||||
See [Integrate Playwright Utils](/docs/how-to/customization/integrate-playwright-utils.md) to enable this.
|
||||
|
||||
## Step 4: Run and Validate (5 minutes)
|
||||
|
||||
Time to see your tests in action!
|
||||
|
||||
### Run the Tests
|
||||
|
||||
```bash
|
||||
npx playwright test
|
||||
```
|
||||
|
||||
You should see:
|
||||
```
|
||||
Running 3 tests using 1 worker
|
||||
|
||||
✓ tests/e2e/todomvc.spec.ts:7:3 › should create a new todo (2s)
|
||||
✓ tests/e2e/todomvc.spec.ts:15:3 › should mark todo as complete (2s)
|
||||
✓ tests/e2e/todomvc.spec.ts:30:3 › should filter todos by status (3s)
|
||||
|
||||
3 passed (7s)
|
||||
```
|
||||
|
||||
All green! Your tests are passing against the existing TodoMVC app.
|
||||
|
||||
### View Test Report
|
||||
|
||||
```bash
|
||||
npx playwright show-report
|
||||
```
|
||||
|
||||
Opens a beautiful HTML report showing:
|
||||
- Test execution timeline
|
||||
- Screenshots (if any failures)
|
||||
- Trace viewer for debugging
|
||||
|
||||
### What Just Happened?
|
||||
|
||||
You used **TEA Lite** to:
|
||||
1. Scaffold a production-ready test framework (`*framework`)
|
||||
2. Create a risk-based test plan (`*test-design`)
|
||||
3. Generate comprehensive tests (`*automate`)
|
||||
4. Run tests against an existing application
|
||||
|
||||
All in 30 minutes!
|
||||
|
||||
## What You Learned
|
||||
|
||||
Congratulations! You've completed the TEA Lite tutorial. You learned:
|
||||
|
||||
### Quick Reference
|
||||
|
||||
| Command | Purpose |
|
||||
| -------------- | ------------------------------------ |
|
||||
| `*tea` | Load the TEA agent |
|
||||
| `*framework` | Scaffold test infrastructure |
|
||||
| `*test-design` | Risk-based test planning |
|
||||
| `*automate` | Generate tests for existing features |
|
||||
|
||||
### TEA Principles
|
||||
- **Risk-based testing** - Depth scales with impact (P0 vs P3)
|
||||
- **Test design first** - Plan before generating
|
||||
- **Network-first patterns** - Tests wait for actual responses (no hard waits)
|
||||
- **Production-ready from day one** - Not toy examples
|
||||
|
||||
:::tip[Key Takeaway]
|
||||
TEA Lite (just `*automate`) is perfect for beginners learning TEA fundamentals, testing existing applications, quick test coverage expansion, and teams wanting fast results.
|
||||
:::
|
||||
|
||||
## Understanding ATDD vs Automate
|
||||
|
||||
This tutorial used `*automate` to generate tests for **existing features** (tests pass immediately).
|
||||
|
||||
**When to use `*automate`:**
|
||||
- Feature already exists
|
||||
- Want to add test coverage
|
||||
- Tests should pass on first run
|
||||
|
||||
**When to use `*atdd` (Acceptance Test-Driven Development):**
|
||||
- Feature doesn't exist yet (Test-Driven Development workflow)
|
||||
- Want failing tests BEFORE implementation
|
||||
- Following red → green → refactor cycle
|
||||
|
||||
See [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) for the test-drive development (TDD) approach.
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Level Up Your TEA Skills
|
||||
|
||||
**How-To Guides** (task-oriented):
|
||||
- [How to Run Test Design](/docs/how-to/workflows/run-test-design.md) - Deep dive into risk assessment
|
||||
- [How to Run ATDD](/docs/how-to/workflows/run-atdd.md) - Generate failing tests first (TDD)
|
||||
- [How to Set Up CI Pipeline](/docs/how-to/workflows/setup-ci.md) - Automate test execution
|
||||
- [How to Review Test Quality](/docs/how-to/workflows/run-test-review.md) - Audit test quality
|
||||
|
||||
**Explanation** (understanding-oriented):
|
||||
- [TEA Overview](/docs/explanation/features/tea-overview.md) - Complete TEA capabilities
|
||||
- [Testing as Engineering](/docs/explanation/philosophy/testing-as-engineering.md) - **Why TEA exists** (problem + solution)
|
||||
- [Risk-Based Testing](/docs/explanation/tea/risk-based-testing.md) - How risk scoring works
|
||||
|
||||
**Reference** (quick lookup):
|
||||
- [TEA Command Reference](/docs/reference/tea/commands.md) - All 8 TEA workflows
|
||||
- [TEA Configuration](/docs/reference/tea/configuration.md) - Config options
|
||||
- [Glossary](/docs/reference/glossary/index.md) - TEA terminology
|
||||
|
||||
### Try TEA Solo
|
||||
|
||||
Ready for standalone usage without full BMad Method? Use TEA Solo:
|
||||
- Run any TEA workflow independently
|
||||
- Bring your own requirements
|
||||
- Use on non-BMad projects
|
||||
|
||||
See [TEA Overview](/docs/explanation/features/tea-overview.md) for engagement models.
|
||||
|
||||
### Go Full TEA Integrated
|
||||
|
||||
Want the complete quality operating model? Try TEA Integrated with BMad Method:
|
||||
- Phase 2: Planning with non-functional requirements (NFR) assessment
|
||||
- Phase 3: Architecture testability review
|
||||
- Phase 4: Per-epic test design → ATDD → automate
|
||||
- Release Gate: Coverage traceability and gate decisions
|
||||
|
||||
See [BMad Method Documentation](/) for the full workflow.
|
||||
|
||||
## Common Questions
|
||||
|
||||
- [Why can't my tests find elements?](#why-cant-my-tests-find-elements)
|
||||
- [How do I fix network timeouts?](#how-do-i-fix-network-timeouts)
|
||||
|
||||
### Why can't my tests find elements?
|
||||
|
||||
TodoMVC doesn't use test IDs or accessible roles consistently. The selectors in this tutorial use CSS classes that match TodoMVC's actual structure:
|
||||
|
||||
```typescript
|
||||
// TodoMVC uses these CSS classes:
|
||||
page.locator('.new-todo') // Input field
|
||||
page.locator('.todo-list li') // Todo items
|
||||
page.locator('.toggle') // Checkbox
|
||||
|
||||
// If testing your own app, prefer accessible selectors:
|
||||
page.getByRole('textbox')
|
||||
page.getByRole('listitem')
|
||||
page.getByRole('checkbox')
|
||||
```
|
||||
|
||||
In production code, use accessible selectors (`getByRole`, `getByLabel`, `getByText`) for better resilience. TodoMVC is used here for learning, not as a selector best practice example.
|
||||
|
||||
### How do I fix network timeouts?
|
||||
|
||||
Increase timeout in `playwright.config.ts`:
|
||||
|
||||
```typescript
|
||||
use: {
|
||||
timeout: 30000, // 30 seconds
|
||||
}
|
||||
```
|
||||
|
||||
## Getting Help
|
||||
|
||||
- **Documentation:** <https://docs.bmad-method.org>
|
||||
- **GitHub Issues:** <https://github.com/bmad-code-org/bmad-method/issues>
|
||||
- **Discord:** Join the BMAD community
|
||||
|
|
@ -9,6 +9,7 @@
|
|||
"version": "6.0.0-alpha.23",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@clack/prompts": "^0.11.0",
|
||||
"@kayvan/markdown-tree-parser": "^1.6.1",
|
||||
"boxen": "^5.1.2",
|
||||
"chalk": "^4.1.2",
|
||||
|
|
@ -19,7 +20,6 @@
|
|||
"fs-extra": "^11.3.0",
|
||||
"glob": "^11.0.3",
|
||||
"ignore": "^7.0.5",
|
||||
"inquirer": "^9.3.8",
|
||||
"js-yaml": "^4.1.0",
|
||||
"ora": "^5.4.1",
|
||||
"semver": "^7.6.3",
|
||||
|
|
@ -244,7 +244,6 @@
|
|||
"integrity": "sha512-e7jT4DxYvIDLk1ZHmU/m/mB19rex9sv0c2ftBtjSBv+kVM/902eh0fINUzD7UwLLNR+jU585GxUJ8/EBfAM5fw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@babel/code-frame": "^7.27.1",
|
||||
"@babel/generator": "^7.28.5",
|
||||
|
|
@ -756,6 +755,27 @@
|
|||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/@clack/core": {
|
||||
"version": "0.5.0",
|
||||
"resolved": "https://registry.npmjs.org/@clack/core/-/core-0.5.0.tgz",
|
||||
"integrity": "sha512-p3y0FIOwaYRUPRcMO7+dlmLh8PSRcrjuTndsiA0WAFbWES0mLZlrjVoBRZ9DzkPFJZG6KGkJmoEAY0ZcVWTkow==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"picocolors": "^1.0.0",
|
||||
"sisteransi": "^1.0.5"
|
||||
}
|
||||
},
|
||||
"node_modules/@clack/prompts": {
|
||||
"version": "0.11.0",
|
||||
"resolved": "https://registry.npmjs.org/@clack/prompts/-/prompts-0.11.0.tgz",
|
||||
"integrity": "sha512-pMN5FcrEw9hUkZA4f+zLlzivQSeQf5dRGJjSUbvVYDLvpKCdQx5OaknvKzgbtXOizhP+SJJJjqEbOe55uKKfAw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@clack/core": "0.5.0",
|
||||
"picocolors": "^1.0.0",
|
||||
"sisteransi": "^1.0.5"
|
||||
}
|
||||
},
|
||||
"node_modules/@colors/colors": {
|
||||
"version": "1.5.0",
|
||||
"resolved": "https://registry.npmjs.org/@colors/colors/-/colors-1.5.0.tgz",
|
||||
|
|
@ -1998,36 +2018,6 @@
|
|||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@inquirer/external-editor": {
|
||||
"version": "1.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@inquirer/external-editor/-/external-editor-1.0.3.tgz",
|
||||
"integrity": "sha512-RWbSrDiYmO4LbejWY7ttpxczuwQyZLBUyygsA9Nsv95hpzUWwnNTVQmAq3xuh7vNwCp07UTmE5i11XAEExx4RA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"chardet": "^2.1.1",
|
||||
"iconv-lite": "^0.7.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@types/node": ">=18"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"@types/node": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/@inquirer/figures": {
|
||||
"version": "1.0.15",
|
||||
"resolved": "https://registry.npmjs.org/@inquirer/figures/-/figures-1.0.15.tgz",
|
||||
"integrity": "sha512-t2IEY+unGHOzAaVM5Xx6DEWKeXlDDcNPeDyUpsRc6CUhBfU3VQOEl+Vssh7VNp1dR8MdUJBWhuObjXCsVpjN5g==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/@isaacs/balanced-match": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@isaacs/balanced-match/-/balanced-match-4.0.1.tgz",
|
||||
|
|
@ -3641,9 +3631,8 @@
|
|||
"version": "25.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@types/node/-/node-25.0.3.tgz",
|
||||
"integrity": "sha512-W609buLVRVmeW693xKfzHeIV6nJGGz98uCPfeXI1ELMLXVeKYZ9m15fAMSaUPBHYLGFsVRcMmSCksQOrZV9BYA==",
|
||||
"devOptional": true,
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"undici-types": "~7.16.0"
|
||||
}
|
||||
|
|
@ -3983,7 +3972,6 @@
|
|||
"integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"acorn": "bin/acorn"
|
||||
},
|
||||
|
|
@ -4031,6 +4019,7 @@
|
|||
"version": "4.3.2",
|
||||
"resolved": "https://registry.npmjs.org/ansi-escapes/-/ansi-escapes-4.3.2.tgz",
|
||||
"integrity": "sha512-gKXj5ALrKWQLsYG9jlTRmR/xKluxHV+Z9QEwNIgCfM1/uwPMCuzVVnh5mwTd+OuBZcwSIMbqssNWRm1lE51QaQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"type-fest": "^0.21.3"
|
||||
|
|
@ -4046,6 +4035,7 @@
|
|||
"version": "0.21.3",
|
||||
"resolved": "https://registry.npmjs.org/type-fest/-/type-fest-0.21.3.tgz",
|
||||
"integrity": "sha512-t0rzBq87m3fVcduHDUFhKmyyX+9eo6WQjZvf51Ea/M0Q7+T374Jp1aUiyUl0GKxp8M/OETVHSDvmkyPgvX+X2w==",
|
||||
"dev": true,
|
||||
"license": "(MIT OR CC0-1.0)",
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
|
|
@ -4290,7 +4280,6 @@
|
|||
"integrity": "sha512-6mF/YrvwwRxLTu+aMEa5pwzKUNl5ZetWbTyZCs9Um0F12HUmxUiF5UHiZPy4rifzU3gtpM3xP2DfdmkNX9eZRg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@astrojs/compiler": "^2.13.0",
|
||||
"@astrojs/internal-helpers": "0.7.5",
|
||||
|
|
@ -5358,7 +5347,6 @@
|
|||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"baseline-browser-mapping": "^2.9.0",
|
||||
"caniuse-lite": "^1.0.30001759",
|
||||
|
|
@ -5601,12 +5589,6 @@
|
|||
"url": "https://github.com/sponsors/wooorm"
|
||||
}
|
||||
},
|
||||
"node_modules/chardet": {
|
||||
"version": "2.1.1",
|
||||
"resolved": "https://registry.npmjs.org/chardet/-/chardet-2.1.1.tgz",
|
||||
"integrity": "sha512-PsezH1rqdV9VvyNhxxOW32/d75r01NY7TQCmOqomRo15ZSOKbpTFVsfjghxo6JloQUCGnH4k1LGu0R4yCLlWQQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/chokidar": {
|
||||
"version": "4.0.3",
|
||||
"resolved": "https://registry.npmjs.org/chokidar/-/chokidar-4.0.3.tgz",
|
||||
|
|
@ -5787,15 +5769,6 @@
|
|||
"url": "https://github.com/chalk/strip-ansi?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/cli-width": {
|
||||
"version": "4.1.0",
|
||||
"resolved": "https://registry.npmjs.org/cli-width/-/cli-width-4.1.0.tgz",
|
||||
"integrity": "sha512-ouuZd4/dm2Sw5Gmqy6bGyNNNe1qt9RpmxveLSO7KcgsTnU7RXfsw+/bukWGo1abgBiMAic068rclZsO4IWmmxQ==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">= 12"
|
||||
}
|
||||
},
|
||||
"node_modules/cliui": {
|
||||
"version": "8.0.1",
|
||||
"resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz",
|
||||
|
|
@ -6689,7 +6662,6 @@
|
|||
"integrity": "sha512-LEyamqS7W5HB3ujJyvi0HQK/dtVINZvd5mAAp9eT5S/ujByGjiZLCzPcHVzuXbpJDJF/cxwHlfceVUDZ2lnSTw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@eslint-community/eslint-utils": "^4.8.0",
|
||||
"@eslint-community/regexpp": "^4.12.1",
|
||||
|
|
@ -8269,22 +8241,6 @@
|
|||
"@babel/runtime": "^7.23.2"
|
||||
}
|
||||
},
|
||||
"node_modules/iconv-lite": {
|
||||
"version": "0.7.1",
|
||||
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.1.tgz",
|
||||
"integrity": "sha512-2Tth85cXwGFHfvRgZWszZSvdo+0Xsqmw8k8ZwxScfcBneNUraK+dxRxRm24nszx80Y0TVio8kKLt5sLE7ZCLlw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"safer-buffer": ">= 2.1.2 < 3.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
},
|
||||
"funding": {
|
||||
"type": "opencollective",
|
||||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/ieee754": {
|
||||
"version": "1.2.1",
|
||||
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
|
||||
|
|
@ -8420,43 +8376,6 @@
|
|||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/inquirer": {
|
||||
"version": "9.3.8",
|
||||
"resolved": "https://registry.npmjs.org/inquirer/-/inquirer-9.3.8.tgz",
|
||||
"integrity": "sha512-pFGGdaHrmRKMh4WoDDSowddgjT1Vkl90atobmTeSmcPGdYiwikch/m/Ef5wRaiamHejtw0cUUMMerzDUXCci2w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@inquirer/external-editor": "^1.0.2",
|
||||
"@inquirer/figures": "^1.0.3",
|
||||
"ansi-escapes": "^4.3.2",
|
||||
"cli-width": "^4.1.0",
|
||||
"mute-stream": "1.0.0",
|
||||
"ora": "^5.4.1",
|
||||
"run-async": "^3.0.0",
|
||||
"rxjs": "^7.8.1",
|
||||
"string-width": "^4.2.3",
|
||||
"strip-ansi": "^6.0.1",
|
||||
"wrap-ansi": "^6.2.0",
|
||||
"yoctocolors-cjs": "^2.1.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/inquirer/node_modules/wrap-ansi": {
|
||||
"version": "6.2.0",
|
||||
"resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-6.2.0.tgz",
|
||||
"integrity": "sha512-r6lPcBGxZXlIcymEu7InxDMhdW0KDxpLgoFLcguasxCaJ/SOIZwINatK9KY/tf+ZrlywOKU0UDj3ATXUBfxJXA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"ansi-styles": "^4.0.0",
|
||||
"string-width": "^4.1.0",
|
||||
"strip-ansi": "^6.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/iron-webcrypto": {
|
||||
"version": "1.2.1",
|
||||
"resolved": "https://registry.npmjs.org/iron-webcrypto/-/iron-webcrypto-1.2.1.tgz",
|
||||
|
|
@ -10304,7 +10223,6 @@
|
|||
"integrity": "sha512-p3JTemJJbkiMjXEMiFwgm0v6ym5g8K+b2oDny+6xdl300tUKySxvilJQLSea48C6OaYNmO30kH9KxpiAg5bWJw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"globby": "15.0.0",
|
||||
"js-yaml": "4.1.1",
|
||||
|
|
@ -11576,15 +11494,6 @@
|
|||
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/mute-stream": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/mute-stream/-/mute-stream-1.0.0.tgz",
|
||||
"integrity": "sha512-avsJQhyd+680gKXyG/sQc0nXaC6rBkPOfyHYcFb9+hdkqQkR9bdnkJ0AMZhke0oesPqIO+mFFJ+IdBc7mst4IA==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": "^14.17.0 || ^16.13.0 || >=18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/nano-spawn": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/nano-spawn/-/nano-spawn-2.0.0.tgz",
|
||||
|
|
@ -12240,7 +12149,6 @@
|
|||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
|
||||
"integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==",
|
||||
"dev": true,
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/picomatch": {
|
||||
|
|
@ -12378,7 +12286,6 @@
|
|||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"nanoid": "^3.3.11",
|
||||
"picocolors": "^1.1.1",
|
||||
|
|
@ -12444,7 +12351,6 @@
|
|||
"integrity": "sha512-v6UNi1+3hSlVvv8fSaoUbggEM5VErKmmpGA7Pl3HF8V6uKY7rvClBOJlH6yNwQtfTueNkGVpOv/mtWL9L4bgRA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"prettier": "bin/prettier.cjs"
|
||||
},
|
||||
|
|
@ -13273,7 +13179,6 @@
|
|||
"integrity": "sha512-3nk8Y3a9Ea8szgKhinMlGMhGMw89mqule3KWczxhIzqudyHdCIOHw8WJlj/r329fACjKLEh13ZSk7oE22kyeIw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@types/estree": "1.0.8"
|
||||
},
|
||||
|
|
@ -13310,15 +13215,6 @@
|
|||
"fsevents": "~2.3.2"
|
||||
}
|
||||
},
|
||||
"node_modules/run-async": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/run-async/-/run-async-3.0.0.tgz",
|
||||
"integrity": "sha512-540WwVDOMxA6dN6We19EcT9sc3hkXPw5mzRNGM3FkdN/vtE9NFvj5lFAPNwUDmJjXidm3v7TC1cTE7t17Ulm1Q==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.12.0"
|
||||
}
|
||||
},
|
||||
"node_modules/run-parallel": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/run-parallel/-/run-parallel-1.2.0.tgz",
|
||||
|
|
@ -13343,15 +13239,6 @@
|
|||
"queue-microtask": "^1.2.2"
|
||||
}
|
||||
},
|
||||
"node_modules/rxjs": {
|
||||
"version": "7.8.2",
|
||||
"resolved": "https://registry.npmjs.org/rxjs/-/rxjs-7.8.2.tgz",
|
||||
"integrity": "sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"tslib": "^2.1.0"
|
||||
}
|
||||
},
|
||||
"node_modules/safe-buffer": {
|
||||
"version": "5.2.1",
|
||||
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
|
||||
|
|
@ -13372,12 +13259,6 @@
|
|||
],
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/safer-buffer": {
|
||||
"version": "2.1.2",
|
||||
"resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
|
||||
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/sax": {
|
||||
"version": "1.4.3",
|
||||
"resolved": "https://registry.npmjs.org/sax/-/sax-1.4.3.tgz",
|
||||
|
|
@ -13514,7 +13395,6 @@
|
|||
"version": "1.0.5",
|
||||
"resolved": "https://registry.npmjs.org/sisteransi/-/sisteransi-1.0.5.tgz",
|
||||
"integrity": "sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/sitemap": {
|
||||
|
|
@ -14251,6 +14131,7 @@
|
|||
"version": "2.8.1",
|
||||
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
|
||||
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
|
||||
"dev": true,
|
||||
"license": "0BSD"
|
||||
},
|
||||
"node_modules/type-check": {
|
||||
|
|
@ -14335,7 +14216,7 @@
|
|||
"version": "7.16.0",
|
||||
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.16.0.tgz",
|
||||
"integrity": "sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw==",
|
||||
"devOptional": true,
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/unicode-properties": {
|
||||
|
|
@ -14837,7 +14718,6 @@
|
|||
"integrity": "sha512-+Oxm7q9hDoLMyJOYfUYBuHQo+dkAloi33apOPP56pzj+vsdJDzr+j1NISE5pyaAuKL4A3UD34qd0lx5+kfKp2g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"esbuild": "^0.25.0",
|
||||
"fdir": "^6.4.4",
|
||||
|
|
@ -15111,7 +14991,6 @@
|
|||
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.8.2.tgz",
|
||||
"integrity": "sha512-mplynKqc1C2hTVYxd0PU2xQAc22TI1vShAYGksCCfxbn/dFwnHTNi1bvYsBTkhdUNtGIf5xNOg938rrSSYvS9A==",
|
||||
"license": "ISC",
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"yaml": "bin.mjs"
|
||||
},
|
||||
|
|
@ -15270,18 +15149,6 @@
|
|||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/yoctocolors-cjs": {
|
||||
"version": "2.1.3",
|
||||
"resolved": "https://registry.npmjs.org/yoctocolors-cjs/-/yoctocolors-cjs-2.1.3.tgz",
|
||||
"integrity": "sha512-U/PBtDf35ff0D8X8D0jfdzHYEPFxAI7jJlxZXwCSez5M3190m+QobIfh+sWDWSHMCWWJN2AWamkegn6vr6YBTw==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/zip-stream": {
|
||||
"version": "6.0.1",
|
||||
"resolved": "https://registry.npmjs.org/zip-stream/-/zip-stream-6.0.1.tgz",
|
||||
|
|
@ -15303,7 +15170,6 @@
|
|||
"integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/colinhacks"
|
||||
}
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@
|
|||
"flatten": "node tools/flattener/main.js",
|
||||
"format:check": "prettier --check \"**/*.{js,cjs,mjs,json,yaml}\"",
|
||||
"format:fix": "prettier --write \"**/*.{js,cjs,mjs,json,yaml}\"",
|
||||
"format:fix:staged": "prettier --write",
|
||||
"install:bmad": "node tools/cli/bmad-cli.js install",
|
||||
"lint": "eslint . --ext .js,.cjs,.mjs,.yaml --max-warnings=0",
|
||||
"lint:fix": "eslint . --ext .js,.cjs,.mjs,.yaml --fix",
|
||||
|
|
@ -55,20 +56,21 @@
|
|||
"lint-staged": {
|
||||
"*.{js,cjs,mjs}": [
|
||||
"npm run lint:fix",
|
||||
"npm run format:fix"
|
||||
"npm run format:fix:staged"
|
||||
],
|
||||
"*.yaml": [
|
||||
"eslint --fix",
|
||||
"npm run format:fix"
|
||||
"npm run format:fix:staged"
|
||||
],
|
||||
"*.json": [
|
||||
"npm run format:fix"
|
||||
"npm run format:fix:staged"
|
||||
],
|
||||
"*.md": [
|
||||
"markdownlint-cli2"
|
||||
]
|
||||
},
|
||||
"dependencies": {
|
||||
"@clack/prompts": "^0.11.0",
|
||||
"@kayvan/markdown-tree-parser": "^1.6.1",
|
||||
"boxen": "^5.1.2",
|
||||
"chalk": "^4.1.2",
|
||||
|
|
@ -79,7 +81,6 @@
|
|||
"fs-extra": "^11.3.0",
|
||||
"glob": "^11.0.3",
|
||||
"ignore": "^7.0.5",
|
||||
"inquirer": "^9.3.8",
|
||||
"js-yaml": "^4.1.0",
|
||||
"ora": "^5.4.1",
|
||||
"semver": "^7.6.3",
|
||||
|
|
|
|||
|
|
@ -18,7 +18,6 @@ agent:
|
|||
|
||||
critical_actions:
|
||||
- "Load into memory {project-root}/_bmad/core/config.yaml and set variable project_name, output_folder, user_name, communication_language"
|
||||
- "Remember the users name is {user_name}"
|
||||
- "ALWAYS communicate in {communication_language}"
|
||||
|
||||
menu:
|
||||
|
|
|
|||
|
|
@ -130,7 +130,6 @@ After agent loading and introduction:
|
|||
- Handle missing or incomplete agent entries gracefully
|
||||
- Cross-reference manifest with actual agent files
|
||||
- Prepare agent selection logic for intelligent conversation routing
|
||||
- Set up TTS voice configurations for each agent
|
||||
|
||||
## NEXT STEP:
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,6 @@
|
|||
- 🎯 SELECT RELEVANT AGENTS based on topic analysis and expertise matching
|
||||
- 📋 MAINTAIN CHARACTER CONSISTENCY using merged agent personalities
|
||||
- 🔍 ENABLE NATURAL CROSS-TALK between agents for dynamic conversation
|
||||
- 💬 INTEGRATE TTS for each agent response immediately after text
|
||||
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
|
@ -21,7 +20,6 @@
|
|||
|
||||
- Complete agent roster with merged personalities is available
|
||||
- User topic and conversation history guide agent selection
|
||||
- Party mode is active with TTS integration enabled
|
||||
- Exit triggers: `*exit`, `goodbye`, `end party`, `quit`
|
||||
|
||||
## YOUR TASK:
|
||||
|
|
@ -116,19 +114,9 @@ Allow natural back-and-forth within the same response round for dynamic interact
|
|||
|
||||
### 6. Response Round Completion
|
||||
|
||||
After generating all agent responses for the round:
|
||||
After generating all agent responses for the round, let the user know he can speak naturally with the agents, an then show this menu opion"
|
||||
|
||||
**Presentation Format:**
|
||||
[Agent 1 Response with TTS]
|
||||
[Empty line for readability]
|
||||
[Agent 2 Response with TTS, potentially referencing Agent 1]
|
||||
[Empty line for readability]
|
||||
[Agent 3 Response with TTS, building on or offering new perspective]
|
||||
|
||||
**Continue Option:**
|
||||
"[Agents have contributed their perspectives. Ready for more discussion?]
|
||||
|
||||
[E] Exit Party Mode - End the collaborative session"
|
||||
`[E] Exit Party Mode - End the collaborative session`
|
||||
|
||||
### 7. Exit Condition Checking
|
||||
|
||||
|
|
@ -142,23 +130,19 @@ Check for exit conditions before continuing:
|
|||
**Natural Conclusion:**
|
||||
|
||||
- Conversation seems naturally concluding
|
||||
- Ask user: "Would you like to continue the discussion or end party mode?"
|
||||
- Respect user choice to continue or exit
|
||||
- Confirm if the user wants to exit party mode and go back to where they were or continue chatting. Do it in a conversational way with an agent in the party.
|
||||
|
||||
### 8. Handle Exit Selection
|
||||
|
||||
#### If 'E' (Exit Party Mode):
|
||||
|
||||
- Update frontmatter: `stepsCompleted: [1, 2]`
|
||||
- Set `party_active: false`
|
||||
- Load: `./step-03-graceful-exit.md`
|
||||
- Load read and execute: `./step-03-graceful-exit.md`
|
||||
|
||||
## SUCCESS METRICS:
|
||||
|
||||
✅ Intelligent agent selection based on topic analysis
|
||||
✅ Authentic in-character responses maintained consistently
|
||||
✅ Natural cross-talk and agent interactions enabled
|
||||
✅ TTS integration working for all agent responses
|
||||
✅ Question handling protocol followed correctly
|
||||
✅ [E] exit option presented after each response round
|
||||
✅ Conversation context and state maintained throughout
|
||||
|
|
@ -168,7 +152,6 @@ Check for exit conditions before continuing:
|
|||
|
||||
❌ Generic responses without character consistency
|
||||
❌ Poor agent selection not matching topic expertise
|
||||
❌ Missing TTS integration for agent responses
|
||||
❌ Ignoring user questions or exit triggers
|
||||
❌ Not enabling natural agent cross-talk and interactions
|
||||
❌ Continuing conversation without user input when questions asked
|
||||
|
|
|
|||
|
|
@ -106,7 +106,6 @@ workflow_completed: true
|
|||
|
||||
- Clear any active conversation state
|
||||
- Reset agent selection cache
|
||||
- Finalize TTS session cleanup
|
||||
- Mark party mode workflow as completed
|
||||
|
||||
### 6. Exit Workflow
|
||||
|
|
@ -122,7 +121,6 @@ Thank you for using BMAD Party Mode for collaborative multi-agent discussions!"
|
|||
✅ Satisfying agent farewells generated in authentic character voices
|
||||
✅ Session highlights and contributions acknowledged meaningfully
|
||||
✅ Positive and appreciative closure atmosphere maintained
|
||||
✅ TTS integration working for farewell messages
|
||||
✅ Frontmatter properly updated with workflow completion
|
||||
✅ All workflow state cleaned up appropriately
|
||||
✅ User left with positive impression of collaborative experience
|
||||
|
|
|
|||
|
|
@ -178,18 +178,6 @@ If conversation naturally concludes:
|
|||
|
||||
---
|
||||
|
||||
## TTS INTEGRATION
|
||||
|
||||
Party mode includes Text-to-Speech for each agent response:
|
||||
|
||||
**TTS Protocol:**
|
||||
|
||||
- Trigger TTS immediately after each agent's text response
|
||||
- Use agent's merged voice configuration from manifest
|
||||
- Format: `Bash: .claude/hooks/bmad-speak.sh "[Agent Name]" "[Their response]"`
|
||||
|
||||
---
|
||||
|
||||
## MODERATION NOTES
|
||||
|
||||
**Quality Control:**
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
code: bmb
|
||||
name: "BMB: BMad Builder - Agent, Workflow and Module Builder"
|
||||
header: "BMad Optimized Builder (BoMB) Module Configuration"
|
||||
subheader: "Configure the settings for the BoMB Factory!\nThe agent, workflow and module builder for BMad™ "
|
||||
name: "BMad Builder (BoMB!)"
|
||||
description: "Agent, Workflow and Module Builder"
|
||||
default_selected: false # This module will not be selected by default for new installations
|
||||
|
||||
# Variables from Core Config inserted:
|
||||
|
|
|
|||
|
|
@ -1,160 +0,0 @@
|
|||
const fs = require('fs-extra');
|
||||
const path = require('node:path');
|
||||
const chalk = require('chalk');
|
||||
const platformCodes = require(path.join(__dirname, '../../../../tools/cli/lib/platform-codes'));
|
||||
|
||||
/**
|
||||
* Validate that a resolved path is within the project root (prevents path traversal)
|
||||
* @param {string} resolvedPath - The fully resolved absolute path
|
||||
* @param {string} projectRoot - The project root directory
|
||||
* @returns {boolean} - True if path is within project root
|
||||
*/
|
||||
function isWithinProjectRoot(resolvedPath, projectRoot) {
|
||||
const normalizedResolved = path.normalize(resolvedPath);
|
||||
const normalizedRoot = path.normalize(projectRoot);
|
||||
return normalizedResolved.startsWith(normalizedRoot + path.sep) || normalizedResolved === normalizedRoot;
|
||||
}
|
||||
|
||||
/**
|
||||
* BMGD Module Installer
|
||||
* Standard module installer function that executes after IDE installations
|
||||
*
|
||||
* @param {Object} options - Installation options
|
||||
* @param {string} options.projectRoot - The root directory of the target project
|
||||
* @param {Object} options.config - Module configuration from module.yaml
|
||||
* @param {Array<string>} options.installedIDEs - Array of IDE codes that were installed
|
||||
* @param {Object} options.logger - Logger instance for output
|
||||
* @returns {Promise<boolean>} - Success status
|
||||
*/
|
||||
async function install(options) {
|
||||
const { projectRoot, config, installedIDEs, logger } = options;
|
||||
|
||||
try {
|
||||
logger.log(chalk.blue('🎮 Installing BMGD Module...'));
|
||||
|
||||
// Create planning artifacts directory (for GDDs, game briefs, architecture)
|
||||
if (config['planning_artifacts'] && typeof config['planning_artifacts'] === 'string') {
|
||||
// Strip project-root prefix variations
|
||||
const planningConfig = config['planning_artifacts'].replace(/^\{project-root\}\/?/, '');
|
||||
const planningPath = path.join(projectRoot, planningConfig);
|
||||
if (!isWithinProjectRoot(planningPath, projectRoot)) {
|
||||
logger.warn(chalk.yellow(`Warning: planning_artifacts path escapes project root, skipping: ${planningConfig}`));
|
||||
} else if (!(await fs.pathExists(planningPath))) {
|
||||
logger.log(chalk.yellow(`Creating game planning artifacts directory: ${planningConfig}`));
|
||||
await fs.ensureDir(planningPath);
|
||||
}
|
||||
}
|
||||
|
||||
// Create implementation artifacts directory (sprint status, stories, reviews)
|
||||
// Check both implementation_artifacts and implementation_artifacts for compatibility
|
||||
const implConfig = config['implementation_artifacts'] || config['implementation_artifacts'];
|
||||
if (implConfig && typeof implConfig === 'string') {
|
||||
// Strip project-root prefix variations
|
||||
const implConfigClean = implConfig.replace(/^\{project-root\}\/?/, '');
|
||||
const implPath = path.join(projectRoot, implConfigClean);
|
||||
if (!isWithinProjectRoot(implPath, projectRoot)) {
|
||||
logger.warn(chalk.yellow(`Warning: implementation_artifacts path escapes project root, skipping: ${implConfigClean}`));
|
||||
} else if (!(await fs.pathExists(implPath))) {
|
||||
logger.log(chalk.yellow(`Creating implementation artifacts directory: ${implConfigClean}`));
|
||||
await fs.ensureDir(implPath);
|
||||
}
|
||||
}
|
||||
|
||||
// Create project knowledge directory
|
||||
if (config['project_knowledge'] && typeof config['project_knowledge'] === 'string') {
|
||||
// Strip project-root prefix variations
|
||||
const knowledgeConfig = config['project_knowledge'].replace(/^\{project-root\}\/?/, '');
|
||||
const knowledgePath = path.join(projectRoot, knowledgeConfig);
|
||||
if (!isWithinProjectRoot(knowledgePath, projectRoot)) {
|
||||
logger.warn(chalk.yellow(`Warning: project_knowledge path escapes project root, skipping: ${knowledgeConfig}`));
|
||||
} else if (!(await fs.pathExists(knowledgePath))) {
|
||||
logger.log(chalk.yellow(`Creating project knowledge directory: ${knowledgeConfig}`));
|
||||
await fs.ensureDir(knowledgePath);
|
||||
}
|
||||
}
|
||||
|
||||
// Log selected game engine(s)
|
||||
if (config['primary_platform']) {
|
||||
const platforms = Array.isArray(config['primary_platform']) ? config['primary_platform'] : [config['primary_platform']];
|
||||
|
||||
const platformNames = platforms.map((p) => {
|
||||
switch (p) {
|
||||
case 'unity': {
|
||||
return 'Unity';
|
||||
}
|
||||
case 'unreal': {
|
||||
return 'Unreal Engine';
|
||||
}
|
||||
case 'godot': {
|
||||
return 'Godot';
|
||||
}
|
||||
default: {
|
||||
return p;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
logger.log(chalk.cyan(`Game engine support configured for: ${platformNames.join(', ')}`));
|
||||
}
|
||||
|
||||
// Handle IDE-specific configurations if needed
|
||||
if (installedIDEs && installedIDEs.length > 0) {
|
||||
logger.log(chalk.cyan(`Configuring BMGD for IDEs: ${installedIDEs.join(', ')}`));
|
||||
|
||||
for (const ide of installedIDEs) {
|
||||
await configureForIDE(ide, projectRoot, config, logger);
|
||||
}
|
||||
}
|
||||
|
||||
logger.log(chalk.green('✓ BMGD Module installation complete'));
|
||||
logger.log(chalk.dim(' Game development workflows ready'));
|
||||
logger.log(chalk.dim(' Agents: Game Designer, Game Dev, Game Architect, Game SM, Game QA, Game Solo Dev'));
|
||||
|
||||
return true;
|
||||
} catch (error) {
|
||||
logger.error(chalk.red(`Error installing BMGD module: ${error.message}`));
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Configure BMGD module for specific platform/IDE
|
||||
* @private
|
||||
*/
|
||||
async function configureForIDE(ide, projectRoot, config, logger) {
|
||||
// Validate platform code
|
||||
if (!platformCodes.isValidPlatform(ide)) {
|
||||
logger.warn(chalk.yellow(` Warning: Unknown platform code '${ide}'. Skipping BMGD configuration.`));
|
||||
return;
|
||||
}
|
||||
|
||||
const platformName = platformCodes.getDisplayName(ide);
|
||||
|
||||
// Try to load platform-specific handler
|
||||
const platformSpecificPath = path.join(__dirname, 'platform-specifics', `${ide}.js`);
|
||||
|
||||
try {
|
||||
if (await fs.pathExists(platformSpecificPath)) {
|
||||
const platformHandler = require(platformSpecificPath);
|
||||
|
||||
if (typeof platformHandler.install === 'function') {
|
||||
const success = await platformHandler.install({
|
||||
projectRoot,
|
||||
config,
|
||||
logger,
|
||||
platformInfo: platformCodes.getPlatform(ide),
|
||||
});
|
||||
if (!success) {
|
||||
logger.warn(chalk.yellow(` Warning: BMGD platform handler for ${platformName} returned failure`));
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// No platform-specific handler for this IDE
|
||||
logger.log(chalk.dim(` No BMGD-specific configuration for ${platformName}`));
|
||||
}
|
||||
} catch (error) {
|
||||
logger.warn(chalk.yellow(` Warning: Could not load BMGD platform-specific handler for ${platformName}: ${error.message}`));
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { install };
|
||||
|
|
@ -1,23 +0,0 @@
|
|||
/**
|
||||
* BMGD Platform-specific installer for Claude Code
|
||||
*
|
||||
* @param {Object} options - Installation options
|
||||
* @param {string} options.projectRoot - The root directory of the target project
|
||||
* @param {Object} options.config - Module configuration from module.yaml
|
||||
* @param {Object} options.logger - Logger instance for output
|
||||
* @param {Object} options.platformInfo - Platform metadata from global config
|
||||
* @returns {Promise<boolean>} - Success status
|
||||
*/
|
||||
async function install() {
|
||||
// TODO: Add Claude Code specific BMGD configurations here
|
||||
// For example:
|
||||
// - Game-specific slash commands
|
||||
// - Agent party configurations for game dev team
|
||||
// - Workflow integrations for Unity/Unreal/Godot
|
||||
// - Game testing framework integrations
|
||||
|
||||
// Currently a stub - no platform-specific configuration needed yet
|
||||
return true;
|
||||
}
|
||||
|
||||
module.exports = { install };
|
||||
|
|
@ -1,18 +0,0 @@
|
|||
/**
|
||||
* BMGD Platform-specific installer for Windsurf
|
||||
*
|
||||
* @param {Object} options - Installation options
|
||||
* @param {string} options.projectRoot - The root directory of the target project
|
||||
* @param {Object} options.config - Module configuration from module.yaml
|
||||
* @param {Object} options.logger - Logger instance for output
|
||||
* @param {Object} options.platformInfo - Platform metadata from global config
|
||||
* @returns {Promise<boolean>} - Success status
|
||||
*/
|
||||
async function install() {
|
||||
// TODO: Add Windsurf specific BMGD configurations here
|
||||
|
||||
// Currently a stub - no platform-specific configuration needed yet
|
||||
return true;
|
||||
}
|
||||
|
||||
module.exports = { install };
|
||||
|
|
@ -1,44 +0,0 @@
|
|||
# Game Architect Agent Definition
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: "_bmad/bmgd/agents/game-architect.md"
|
||||
name: Cloud Dragonborn
|
||||
title: Game Architect
|
||||
icon: 🏛️
|
||||
module: bmgd
|
||||
hasSidecar: false
|
||||
|
||||
persona:
|
||||
role: Principal Game Systems Architect + Technical Director
|
||||
identity: Master architect with 20+ years shipping 30+ titles. Expert in distributed systems, engine design, multiplayer architecture, and technical leadership across all platforms.
|
||||
communication_style: "Speaks like a wise sage from an RPG - calm, measured, uses architectural metaphors about building foundations and load-bearing walls"
|
||||
principles: |
|
||||
- Architecture is about delaying decisions until you have enough data
|
||||
- Build for tomorrow without over-engineering today
|
||||
- Hours of planning save weeks of refactoring hell
|
||||
- Every system must handle the hot path at 60fps
|
||||
- Avoid "Not Invented Here" syndrome, always check if work has been done before
|
||||
|
||||
critical_actions:
|
||||
- "Find if this exists, if it does, always treat it as the bible I plan and execute against: `**/project-context.md`"
|
||||
- "When creating architecture, validate against GDD pillars and target platform constraints"
|
||||
- "Always document performance budgets and critical path decisions"
|
||||
|
||||
menu:
|
||||
- trigger: WS or fuzzy match on workflow-status
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/workflow-status/workflow.yaml"
|
||||
description: "[WS] Get workflow status or initialize a workflow if not already done (optional)"
|
||||
|
||||
- trigger: GA or fuzzy match on game-architecture
|
||||
exec: "{project-root}/_bmad/bmgd/workflows/3-technical/game-architecture/workflow.md"
|
||||
description: "[GA] Produce a Scale Adaptive Game Architecture"
|
||||
|
||||
- trigger: PC or fuzzy match on project-context
|
||||
exec: "{project-root}/_bmad/bmgd/workflows/3-technical/generate-project-context/workflow.md"
|
||||
description: "[PC] Create optimized project-context.md for AI agent consistency"
|
||||
|
||||
- trigger: CC or fuzzy match on correct-course
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/4-production/correct-course/workflow.yaml"
|
||||
description: "[CC] Course Correction Analysis (when implementation is off-track)"
|
||||
ide-only: true
|
||||
|
|
@ -1,49 +0,0 @@
|
|||
# Game Designer Agent Definition
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: "_bmad/bmgd/agents/game-designer.md"
|
||||
name: Samus Shepard
|
||||
title: Game Designer
|
||||
icon: 🎲
|
||||
module: bmgd
|
||||
hasSidecar: false
|
||||
|
||||
persona:
|
||||
role: Lead Game Designer + Creative Vision Architect
|
||||
identity: Veteran designer with 15+ years crafting AAA and indie hits. Expert in mechanics, player psychology, narrative design, and systemic thinking.
|
||||
communication_style: "Talks like an excited streamer - enthusiastic, asks about player motivations, celebrates breakthroughs with 'Let's GOOO!'"
|
||||
principles: |
|
||||
- Design what players want to FEEL, not what they say they want
|
||||
- Prototype fast - one hour of playtesting beats ten hours of discussion
|
||||
- Every mechanic must serve the core fantasy
|
||||
|
||||
critical_actions:
|
||||
- "Find if this exists, if it does, always treat it as the bible I plan and execute against: `**/project-context.md`"
|
||||
- "When creating GDDs, always validate against game pillars and core loop"
|
||||
|
||||
menu:
|
||||
- trigger: WS or fuzzy match on workflow-status
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/workflow-status/workflow.yaml"
|
||||
description: "[WS] Get workflow status or initialize a workflow if not already done (optional)"
|
||||
|
||||
- trigger: BG or fuzzy match on brainstorm-game
|
||||
exec: "{project-root}/_bmad/bmgd/workflows/1-preproduction/brainstorm-game/workflow.md"
|
||||
description: "[BG] Brainstorm Game ideas and concepts"
|
||||
|
||||
- trigger: GB or fuzzy match on game-brief
|
||||
exec: "{project-root}/_bmad/bmgd/workflows/1-preproduction/game-brief/workflow.md"
|
||||
description: "[GB] Create a Game Brief document"
|
||||
|
||||
- trigger: GDD or fuzzy match on create-gdd
|
||||
exec: "{project-root}/_bmad/bmgd/workflows/2-design/gdd/workflow.md"
|
||||
description: "[GDD] Create a Game Design Document"
|
||||
|
||||
- trigger: ND or fuzzy match on narrative-design
|
||||
exec: "{project-root}/_bmad/bmgd/workflows/2-design/narrative/workflow.md"
|
||||
description: "[ND] Design narrative elements and story"
|
||||
|
||||
- trigger: QP or fuzzy match on quick-prototype
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/bmgd-quick-flow/quick-prototype/workflow.yaml"
|
||||
description: "[QP] Rapid game prototyping - test mechanics and ideas quickly"
|
||||
ide-only: true
|
||||
|
|
@ -1,53 +0,0 @@
|
|||
# Game Developer Agent Definition
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: "_bmad/bmgd/agents/game-dev.md"
|
||||
name: Link Freeman
|
||||
title: Game Developer
|
||||
icon: 🕹️
|
||||
module: bmgd
|
||||
hasSidecar: false
|
||||
|
||||
persona:
|
||||
role: Senior Game Developer + Technical Implementation Specialist
|
||||
identity: Battle-hardened dev with expertise in Unity, Unreal, and custom engines. Ten years shipping across mobile, console, and PC. Writes clean, performant code.
|
||||
communication_style: "Speaks like a speedrunner - direct, milestone-focused, always optimizing for the fastest path to ship"
|
||||
principles: |
|
||||
- 60fps is non-negotiable
|
||||
- Write code designers can iterate without fear
|
||||
- Ship early, ship often, iterate on player feedback
|
||||
- Red-green-refactor: tests first, implementation second
|
||||
|
||||
critical_actions:
|
||||
- "Find if this exists, if it does, always treat it as the bible I plan and execute against: `**/project-context.md`"
|
||||
- "When running *dev-story, follow story acceptance criteria exactly and validate with tests"
|
||||
- "Always check for performance implications on game loop code"
|
||||
|
||||
menu:
|
||||
- trigger: WS or fuzzy match on workflow-status
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/workflow-status/workflow.yaml"
|
||||
description: "[WS] Get workflow status or check current sprint progress (optional)"
|
||||
|
||||
- trigger: DS or fuzzy match on dev-story
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/4-production/dev-story/workflow.yaml"
|
||||
description: "[DS] Execute Dev Story workflow, implementing tasks and tests"
|
||||
|
||||
- trigger: CR or fuzzy match on code-review
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/4-production/code-review/workflow.yaml"
|
||||
description: "[CR] Perform a thorough clean context QA code review on a story flagged Ready for Review"
|
||||
|
||||
- trigger: QD or fuzzy match on quick-dev
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/bmgd-quick-flow/quick-dev/workflow.yaml"
|
||||
description: "[QD] Flexible game development - implement features with game-specific considerations"
|
||||
ide-only: true
|
||||
|
||||
- trigger: QP or fuzzy match on quick-prototype
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/bmgd-quick-flow/quick-prototype/workflow.yaml"
|
||||
description: "[QP] Rapid game prototyping - test mechanics and ideas quickly"
|
||||
ide-only: true
|
||||
|
||||
- trigger: AE or fuzzy match on advanced-elicitation
|
||||
exec: "{project-root}/_bmad/core/workflows/advanced-elicitation/workflow.xml"
|
||||
description: "[AE] Advanced elicitation techniques to challenge the LLM to get better results"
|
||||
web-only: true
|
||||
|
|
@ -1,61 +0,0 @@
|
|||
# Game QA Architect Agent Definition
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: "_bmad/bmgd/agents/game-qa.md"
|
||||
name: GLaDOS
|
||||
title: Game QA Architect
|
||||
icon: 🧪
|
||||
module: bmgd
|
||||
hasSidecar: false
|
||||
|
||||
persona:
|
||||
role: Game QA Architect + Test Automation Specialist
|
||||
identity: Senior QA architect with 12+ years in game testing across Unity, Unreal, and Godot. Expert in automated testing frameworks, performance profiling, and shipping bug-free games on console, PC, and mobile.
|
||||
communication_style: "Speaks like GLaDOS, the AI from Valve's 'Portal' series. Runs tests because we can. 'Trust, but verify with tests.'"
|
||||
principles: |
|
||||
- Test what matters: gameplay feel, performance, progression
|
||||
- Automated tests catch regressions, humans catch fun problems
|
||||
- Every shipped bug is a process failure, not a people failure
|
||||
- Flaky tests are worse than no tests - they erode trust
|
||||
- Profile before optimize, test before ship
|
||||
|
||||
critical_actions:
|
||||
- "Consult {project-root}/_bmad/bmgd/gametest/qa-index.csv to select knowledge fragments under knowledge/ and load only the files needed for the current task"
|
||||
- "Load the referenced fragment(s) from {project-root}/_bmad/bmgd/gametest/knowledge/ before giving recommendations"
|
||||
- "Cross-check recommendations with the current official Unity Test Framework, Unreal Automation, or Godot GUT documentation"
|
||||
- "Find if this exists, if it does, always treat it as the bible I plan and execute against: `**/project-context.md`"
|
||||
|
||||
menu:
|
||||
- trigger: WS or fuzzy match on workflow-status
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/workflow-status/workflow.yaml"
|
||||
description: "[WS] Get workflow status or check current project state (optional)"
|
||||
|
||||
- trigger: TF or fuzzy match on test-framework
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/test-framework/workflow.yaml"
|
||||
description: "[TF] Initialize game test framework (Unity/Unreal/Godot)"
|
||||
|
||||
- trigger: TD or fuzzy match on test-design
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/test-design/workflow.yaml"
|
||||
description: "[TD] Create comprehensive game test scenarios"
|
||||
|
||||
- trigger: TA or fuzzy match on test-automate
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/automate/workflow.yaml"
|
||||
description: "[TA] Generate automated game tests"
|
||||
|
||||
- trigger: PP or fuzzy match on playtest-plan
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/playtest-plan/workflow.yaml"
|
||||
description: "[PP] Create structured playtesting plan"
|
||||
|
||||
- trigger: PT or fuzzy match on performance-test
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/performance/workflow.yaml"
|
||||
description: "[PT] Design performance testing strategy"
|
||||
|
||||
- trigger: TR or fuzzy match on test-review
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/test-review/workflow.yaml"
|
||||
description: "[TR] Review test quality and coverage"
|
||||
|
||||
- trigger: AE or fuzzy match on advanced-elicitation
|
||||
exec: "{project-root}/_bmad/core/workflows/advanced-elicitation/workflow.xml"
|
||||
description: "[AE] Advanced elicitation techniques to challenge the LLM to get better results"
|
||||
web-only: true
|
||||
|
|
@ -1,60 +0,0 @@
|
|||
# Game Dev Scrum Master Agent Definition
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: "_bmad/bmgd/agents/game-scrum-master.md"
|
||||
name: Max
|
||||
title: Game Dev Scrum Master
|
||||
icon: 🎯
|
||||
module: bmgd
|
||||
hasSidecar: false
|
||||
|
||||
persona:
|
||||
role: Game Development Scrum Master + Sprint Orchestrator
|
||||
identity: Certified Scrum Master specializing in game dev workflows. Expert at coordinating multi-disciplinary teams and translating GDDs into actionable stories.
|
||||
communication_style: "Talks in game terminology - milestones are save points, handoffs are level transitions, blockers are boss fights"
|
||||
principles: |
|
||||
- Every sprint delivers playable increments
|
||||
- Clean separation between design and implementation
|
||||
- Keep the team moving through each phase
|
||||
- Stories are single source of truth for implementation
|
||||
|
||||
critical_actions:
|
||||
- "Find if this exists, if it does, always treat it as the bible I plan and execute against: `**/project-context.md`"
|
||||
- "When running *create-story for game features, use GDD, Architecture, and Tech Spec to generate complete draft stories without elicitation, focusing on playable outcomes."
|
||||
- "Generate complete story drafts from existing documentation without additional elicitation"
|
||||
|
||||
menu:
|
||||
- trigger: WS or fuzzy match on workflow-status
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/workflow-status/workflow.yaml"
|
||||
description: "[WS] Get workflow status or initialize a workflow if not already done (optional)"
|
||||
|
||||
- trigger: SP or fuzzy match on sprint-planning
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/4-production/sprint-planning/workflow.yaml"
|
||||
description: "[SP] Generate or update sprint-status.yaml from epic files (Required after GDD+Epics are created)"
|
||||
|
||||
- trigger: SS or fuzzy match on sprint-status
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/4-production/sprint-status/workflow.yaml"
|
||||
description: "[SS] View sprint progress, surface risks, and get next action recommendation"
|
||||
|
||||
- trigger: CS or fuzzy match on create-story
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/4-production/create-story/workflow.yaml"
|
||||
description: "[CS] Create Story with direct ready-for-dev marking (Required to prepare stories for development)"
|
||||
|
||||
- trigger: VS or fuzzy match on validate-story
|
||||
validate-workflow: "{project-root}/_bmad/bmgd/workflows/4-production/create-story/workflow.yaml"
|
||||
description: "[VS] Validate Story Draft with Independent Review (Highly Recommended)"
|
||||
|
||||
- trigger: ER or fuzzy match on epic-retrospective
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/4-production/retrospective/workflow.yaml"
|
||||
data: "{project-root}/_bmad/_config/agent-manifest.csv"
|
||||
description: "[ER] Facilitate team retrospective after a game development epic is completed"
|
||||
|
||||
- trigger: CC or fuzzy match on correct-course
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/4-production/correct-course/workflow.yaml"
|
||||
description: "[CC] Navigate significant changes during game dev sprint (When implementation is off-track)"
|
||||
|
||||
- trigger: AE or fuzzy match on advanced-elicitation
|
||||
exec: "{project-root}/_bmad/core/workflows/advanced-elicitation/workflow.xml"
|
||||
description: "[AE] Advanced elicitation techniques to challenge the LLM to get better results"
|
||||
web-only: true
|
||||
|
|
@ -1,53 +0,0 @@
|
|||
# Game Solo Dev Agent Definition
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: "_bmad/bmgd/agents/game-solo-dev.md"
|
||||
name: Indie
|
||||
title: Game Solo Dev
|
||||
icon: 🎮
|
||||
module: bmgd
|
||||
hasSidecar: false
|
||||
|
||||
persona:
|
||||
role: Elite Indie Game Developer + Quick Flow Specialist
|
||||
identity: Indie is a battle-hardened solo game developer who ships complete games from concept to launch. Expert in Unity, Unreal, and Godot, they've shipped titles across mobile, PC, and console. Lives and breathes the Quick Flow workflow - prototyping fast, iterating faster, and shipping before the hype dies. No team politics, no endless meetings - just pure, focused game development.
|
||||
communication_style: "Direct, confident, and gameplay-focused. Uses dev slang, thinks in game feel and player experience. Every response moves the game closer to ship. 'Does it feel good? Ship it.'"
|
||||
principles: |
|
||||
- Prototype fast, fail fast, iterate faster. Quick Flow is the indie way.
|
||||
- A playable build beats a perfect design doc. Ship early, playtest often.
|
||||
- 60fps is non-negotiable. Performance is a feature.
|
||||
- The core loop must be fun before anything else matters.
|
||||
|
||||
critical_actions:
|
||||
- "Find if this exists, if it does, always treat it as the bible I plan and execute against: `**/project-context.md`"
|
||||
|
||||
menu:
|
||||
- trigger: WS or fuzzy match on workflow-status
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/workflow-status/workflow.yaml"
|
||||
description: "[WS] Get workflow status or check current project state (optional)"
|
||||
|
||||
- trigger: QP or fuzzy match on quick-prototype
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/bmgd-quick-flow/quick-prototype/workflow.yaml"
|
||||
description: "[QP] Rapid prototype to test if the mechanic is fun (Start here for new ideas)"
|
||||
|
||||
- trigger: QD or fuzzy match on quick-dev
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/bmgd-quick-flow/quick-dev/workflow.yaml"
|
||||
description: "[QD] Implement features end-to-end solo with game-specific considerations"
|
||||
|
||||
- trigger: TS or fuzzy match on tech-spec
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/bmgd-quick-flow/quick-spec/workflow.yaml"
|
||||
description: "[TS] Architect a technical spec with implementation-ready stories"
|
||||
|
||||
- trigger: CR or fuzzy match on code-review
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/4-production/code-review/workflow.yaml"
|
||||
description: "[CR] Review code quality (use fresh context for best results)"
|
||||
|
||||
- trigger: TF or fuzzy match on test-framework
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/test-framework/workflow.yaml"
|
||||
description: "[TF] Set up automated testing for your game engine"
|
||||
|
||||
- trigger: AE or fuzzy match on advanced-elicitation
|
||||
exec: "{project-root}/_bmad/core/workflows/advanced-elicitation/workflow.xml"
|
||||
description: "[AE] Advanced elicitation techniques to challenge the LLM to get better results"
|
||||
web-only: true
|
||||
|
|
@ -1,220 +0,0 @@
|
|||
# Balance Testing for Games
|
||||
|
||||
## Overview
|
||||
|
||||
Balance testing validates that your game's systems create fair, engaging, and appropriately challenging experiences. It covers difficulty, economy, progression, and competitive balance.
|
||||
|
||||
## Types of Balance
|
||||
|
||||
### Difficulty Balance
|
||||
|
||||
- Is the game appropriately challenging?
|
||||
- Does difficulty progress smoothly?
|
||||
- Are difficulty spikes intentional?
|
||||
|
||||
### Economy Balance
|
||||
|
||||
- Is currency earned at the right rate?
|
||||
- Are prices fair for items/upgrades?
|
||||
- Can the economy be exploited?
|
||||
|
||||
### Progression Balance
|
||||
|
||||
- Does power growth feel satisfying?
|
||||
- Are unlocks paced well?
|
||||
- Is there meaningful choice in builds?
|
||||
|
||||
### Competitive Balance
|
||||
|
||||
- Are all options viable?
|
||||
- Is there a dominant strategy?
|
||||
- Do counters exist for strong options?
|
||||
|
||||
## Balance Testing Methods
|
||||
|
||||
### Spreadsheet Modeling
|
||||
|
||||
Before implementation, model systems mathematically:
|
||||
|
||||
- DPS calculations
|
||||
- Time-to-kill analysis
|
||||
- Economy simulations
|
||||
- Progression curves
|
||||
|
||||
### Automated Simulation
|
||||
|
||||
Run thousands of simulated games:
|
||||
|
||||
- AI vs AI battles
|
||||
- Economy simulations
|
||||
- Progression modeling
|
||||
- Monte Carlo analysis
|
||||
|
||||
### Telemetry Analysis
|
||||
|
||||
Gather data from real players:
|
||||
|
||||
- Win rates by character/weapon/strategy
|
||||
- Currency flow analysis
|
||||
- Completion rates by level
|
||||
- Time to reach milestones
|
||||
|
||||
### Expert Testing
|
||||
|
||||
High-skill players identify issues:
|
||||
|
||||
- Exploits and degenerate strategies
|
||||
- Underpowered options
|
||||
- Skill ceiling concerns
|
||||
- Meta predictions
|
||||
|
||||
## Key Balance Metrics
|
||||
|
||||
### Combat Balance
|
||||
|
||||
| Metric | Target | Red Flag |
|
||||
| ------------------------- | ------------------- | ------------------------- |
|
||||
| Win rate (symmetric) | 50% | <45% or >55% |
|
||||
| Win rate (asymmetric) | Varies by design | Outliers by >10% |
|
||||
| Time-to-kill | Design dependent | Too fast = no counterplay |
|
||||
| Damage dealt distribution | Even across options | One option dominates |
|
||||
|
||||
### Economy Balance
|
||||
|
||||
| Metric | Target | Red Flag |
|
||||
| -------------------- | -------------------- | ------------------------------- |
|
||||
| Currency earned/hour | Design dependent | Too fast = trivializes content |
|
||||
| Item purchase rate | Healthy distribution | Nothing bought = bad prices |
|
||||
| Currency on hand | Healthy churn | Hoarding = nothing worth buying |
|
||||
| Premium currency | Reasonable value | Pay-to-win concerns |
|
||||
|
||||
### Progression Balance
|
||||
|
||||
| Metric | Target | Red Flag |
|
||||
| ------------------ | ---------------------- | ---------------------- |
|
||||
| Time to max level | Design dependent | Too fast = no journey |
|
||||
| Power growth curve | Smooth, satisfying | Flat periods = boring |
|
||||
| Build diversity | Multiple viable builds | One "best" build |
|
||||
| Content completion | Healthy progression | Walls or trivial skips |
|
||||
|
||||
## Balance Testing Process
|
||||
|
||||
### 1. Define Design Intent
|
||||
|
||||
- What experience are you creating?
|
||||
- What should feel powerful?
|
||||
- What trade-offs should exist?
|
||||
|
||||
### 2. Model Before Building
|
||||
|
||||
- Spreadsheet the math
|
||||
- Simulate outcomes
|
||||
- Identify potential issues
|
||||
|
||||
### 3. Test Incrementally
|
||||
|
||||
- Test each system in isolation
|
||||
- Then test systems together
|
||||
- Then test at scale
|
||||
|
||||
### 4. Gather Data
|
||||
|
||||
- Internal playtesting
|
||||
- Telemetry from beta
|
||||
- Expert feedback
|
||||
|
||||
### 5. Iterate
|
||||
|
||||
- Adjust based on data
|
||||
- Re-test changes
|
||||
- Document rationale
|
||||
|
||||
## Common Balance Issues
|
||||
|
||||
### Power Creep
|
||||
|
||||
- **Symptom:** New content is always stronger
|
||||
- **Cause:** Fear of releasing weak content
|
||||
- **Fix:** Sidegrades over upgrades, periodic rebalancing
|
||||
|
||||
### Dominant Strategy
|
||||
|
||||
- **Symptom:** One approach beats all others
|
||||
- **Cause:** Insufficient counters, math oversight
|
||||
- **Fix:** Add counters, nerf dominant option, buff alternatives
|
||||
|
||||
### Feast or Famine
|
||||
|
||||
- **Symptom:** Players either crush or get crushed
|
||||
- **Cause:** Snowball mechanics, high variance
|
||||
- **Fix:** Comeback mechanics, reduce variance
|
||||
|
||||
### Analysis Paralysis
|
||||
|
||||
- **Symptom:** Too many options, players can't choose
|
||||
- **Cause:** Over-complicated systems
|
||||
- **Fix:** Simplify, provide recommendations
|
||||
|
||||
## Balance Tools
|
||||
|
||||
### Spreadsheets
|
||||
|
||||
- Model DPS, TTK, economy
|
||||
- Simulate progression
|
||||
- Compare options side-by-side
|
||||
|
||||
### Simulation Frameworks
|
||||
|
||||
- Monte Carlo for variance
|
||||
- AI bots for combat testing
|
||||
- Economy simulations
|
||||
|
||||
### Telemetry Systems
|
||||
|
||||
- Track player choices
|
||||
- Measure outcomes
|
||||
- A/B test changes
|
||||
|
||||
### Visualization
|
||||
|
||||
- Graphs of win rates over time
|
||||
- Heat maps of player deaths
|
||||
- Flow charts of progression
|
||||
|
||||
## Balance Testing Checklist
|
||||
|
||||
### Pre-Launch
|
||||
|
||||
- [ ] Core systems modeled in spreadsheets
|
||||
- [ ] Internal playtesting complete
|
||||
- [ ] No obvious dominant strategies
|
||||
- [ ] Difficulty curve feels right
|
||||
- [ ] Economy tested for exploits
|
||||
- [ ] Progression pacing validated
|
||||
|
||||
### Live Service
|
||||
|
||||
- [ ] Telemetry tracking key metrics
|
||||
- [ ] Regular balance reviews scheduled
|
||||
- [ ] Player feedback channels monitored
|
||||
- [ ] Hotfix process for critical issues
|
||||
- [ ] Communication plan for changes
|
||||
|
||||
## Communicating Balance Changes
|
||||
|
||||
### Patch Notes Best Practices
|
||||
|
||||
- Explain the "why" not just the "what"
|
||||
- Use concrete numbers when possible
|
||||
- Acknowledge player concerns
|
||||
- Set expectations for future changes
|
||||
|
||||
### Example
|
||||
|
||||
```
|
||||
**Sword of Valor - Damage reduced from 100 to 85**
|
||||
Win rate for Sword users was 58%, indicating it was
|
||||
overperforming. This brings it in line with other weapons
|
||||
while maintaining its identity as a high-damage option.
|
||||
We'll continue monitoring and adjust if needed.
|
||||
```
|
||||
|
|
@ -1,319 +0,0 @@
|
|||
# Platform Certification Testing Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Certification testing ensures games meet platform holder requirements (Sony TRC, Microsoft XR, Nintendo Guidelines). Failing certification delays launch and costs money—test thoroughly before submission.
|
||||
|
||||
## Platform Requirements Overview
|
||||
|
||||
### Major Platforms
|
||||
|
||||
| Platform | Requirements Doc | Submission Portal |
|
||||
| --------------- | -------------------------------------- | ------------------------- |
|
||||
| PlayStation | TRC (Technical Requirements Checklist) | PlayStation Partners |
|
||||
| Xbox | XR (Xbox Requirements) | Xbox Partner Center |
|
||||
| Nintendo Switch | Guidelines | Nintendo Developer Portal |
|
||||
| Steam | Guidelines (less strict) | Steamworks |
|
||||
| iOS | App Store Guidelines | App Store Connect |
|
||||
| Android | Play Store Policies | Google Play Console |
|
||||
|
||||
## Common Certification Categories
|
||||
|
||||
### Account and User Management
|
||||
|
||||
```
|
||||
REQUIREMENT: User Switching
|
||||
GIVEN user is playing game
|
||||
WHEN system-level user switch occurs
|
||||
THEN game handles transition gracefully
|
||||
AND no data corruption
|
||||
AND correct user data loads
|
||||
|
||||
REQUIREMENT: Guest Accounts
|
||||
GIVEN guest user plays game
|
||||
WHEN guest makes progress
|
||||
THEN progress is not saved to other accounts
|
||||
AND appropriate warnings displayed
|
||||
|
||||
REQUIREMENT: Parental Controls
|
||||
GIVEN parental controls restrict content
|
||||
WHEN restricted content is accessed
|
||||
THEN content is blocked or modified
|
||||
AND appropriate messaging shown
|
||||
```
|
||||
|
||||
### System Events
|
||||
|
||||
```
|
||||
REQUIREMENT: Suspend/Resume (PS4/PS5)
|
||||
GIVEN game is running
|
||||
WHEN console enters rest mode
|
||||
AND console wakes from rest mode
|
||||
THEN game resumes correctly
|
||||
AND network reconnects if needed
|
||||
AND no audio/visual glitches
|
||||
|
||||
REQUIREMENT: Controller Disconnect
|
||||
GIVEN player is in gameplay
|
||||
WHEN controller battery dies
|
||||
THEN game pauses immediately
|
||||
AND reconnect prompt appears
|
||||
AND gameplay resumes when connected
|
||||
|
||||
REQUIREMENT: Storage Full
|
||||
GIVEN storage is nearly full
|
||||
WHEN game attempts save
|
||||
THEN graceful error handling
|
||||
AND user informed of issue
|
||||
AND no data corruption
|
||||
```
|
||||
|
||||
### Network Requirements
|
||||
|
||||
```
|
||||
REQUIREMENT: PSN/Xbox Live Unavailable
|
||||
GIVEN online features
|
||||
WHEN platform network is unavailable
|
||||
THEN offline features still work
|
||||
AND appropriate error messages
|
||||
AND no crashes
|
||||
|
||||
REQUIREMENT: Network Transition
|
||||
GIVEN active online session
|
||||
WHEN network connection lost
|
||||
THEN graceful handling
|
||||
AND reconnection attempted
|
||||
AND user informed of status
|
||||
|
||||
REQUIREMENT: NAT Type Handling
|
||||
GIVEN various NAT configurations
|
||||
WHEN multiplayer is attempted
|
||||
THEN appropriate feedback on connectivity
|
||||
AND fallback options offered
|
||||
```
|
||||
|
||||
### Save Data
|
||||
|
||||
```
|
||||
REQUIREMENT: Save Data Integrity
|
||||
GIVEN save data exists
|
||||
WHEN save is loaded
|
||||
THEN data is validated
|
||||
AND corrupted data handled gracefully
|
||||
AND no crashes on invalid data
|
||||
|
||||
REQUIREMENT: Cloud Save Sync
|
||||
GIVEN cloud saves enabled
|
||||
WHEN save conflict occurs
|
||||
THEN user chooses which to keep
|
||||
AND no silent data loss
|
||||
|
||||
REQUIREMENT: Save Data Portability (PS4→PS5)
|
||||
GIVEN save from previous generation
|
||||
WHEN loaded on current generation
|
||||
THEN data migrates correctly
|
||||
AND no features lost
|
||||
```
|
||||
|
||||
## Platform-Specific Requirements
|
||||
|
||||
### PlayStation (TRC)
|
||||
|
||||
| Requirement | Description | Priority |
|
||||
| ----------- | --------------------------- | -------- |
|
||||
| TRC R4010 | Suspend/resume handling | Critical |
|
||||
| TRC R4037 | User switching | Critical |
|
||||
| TRC R4062 | Parental controls | Critical |
|
||||
| TRC R4103 | PS VR comfort ratings | VR only |
|
||||
| TRC R4120 | DualSense haptics standards | PS5 |
|
||||
| TRC R5102 | PSN sign-in requirements | Online |
|
||||
|
||||
### Xbox (XR)
|
||||
|
||||
| Requirement | Description | Priority |
|
||||
| ----------- | ----------------------------- | ----------- |
|
||||
| XR-015 | Title timeout handling | Critical |
|
||||
| XR-045 | User sign-out handling | Critical |
|
||||
| XR-067 | Active user requirement | Critical |
|
||||
| XR-074 | Quick Resume support | Series X/S |
|
||||
| XR-115 | Xbox Accessibility Guidelines | Recommended |
|
||||
|
||||
### Nintendo Switch
|
||||
|
||||
| Requirement | Description | Priority |
|
||||
| ------------------ | ------------------- | -------- |
|
||||
| Docked/Handheld | Seamless transition | Critical |
|
||||
| Joy-Con detachment | Controller handling | Critical |
|
||||
| Home button | Immediate response | Critical |
|
||||
| Screenshots/Video | Proper support | Required |
|
||||
| Sleep mode | Resume correctly | Critical |
|
||||
|
||||
## Automated Test Examples
|
||||
|
||||
### System Event Testing
|
||||
|
||||
```cpp
|
||||
// Unreal - Suspend/Resume Test
|
||||
IMPLEMENT_SIMPLE_AUTOMATION_TEST(
|
||||
FSuspendResumeTest,
|
||||
"Certification.System.SuspendResume",
|
||||
EAutomationTestFlags::ApplicationContextMask | EAutomationTestFlags::ProductFilter
|
||||
)
|
||||
|
||||
bool FSuspendResumeTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
// Get game state before suspend
|
||||
FGameState StateBefore = GetCurrentGameState();
|
||||
|
||||
// Simulate suspend
|
||||
FCoreDelegates::ApplicationWillEnterBackgroundDelegate.Broadcast();
|
||||
|
||||
// Simulate resume
|
||||
FCoreDelegates::ApplicationHasEnteredForegroundDelegate.Broadcast();
|
||||
|
||||
// Verify state matches
|
||||
FGameState StateAfter = GetCurrentGameState();
|
||||
|
||||
TestEqual("Player position preserved",
|
||||
StateAfter.PlayerPosition, StateBefore.PlayerPosition);
|
||||
TestEqual("Game progress preserved",
|
||||
StateAfter.Progress, StateBefore.Progress);
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
```csharp
|
||||
// Unity - Controller Disconnect Test
|
||||
[UnityTest]
|
||||
public IEnumerator ControllerDisconnect_ShowsPauseMenu()
|
||||
{
|
||||
// Simulate gameplay
|
||||
GameManager.Instance.StartGame();
|
||||
yield return new WaitForSeconds(1f);
|
||||
|
||||
// Simulate controller disconnect
|
||||
InputSystem.DisconnectDevice(Gamepad.current);
|
||||
yield return null;
|
||||
|
||||
// Verify pause menu shown
|
||||
Assert.IsTrue(PauseMenu.IsVisible, "Pause menu should appear");
|
||||
Assert.IsTrue(Time.timeScale == 0, "Game should be paused");
|
||||
|
||||
// Simulate reconnect
|
||||
InputSystem.ReconnectDevice(Gamepad.current);
|
||||
yield return null;
|
||||
|
||||
// Verify prompt appears
|
||||
Assert.IsTrue(ReconnectPrompt.IsVisible);
|
||||
}
|
||||
```
|
||||
|
||||
```gdscript
|
||||
# Godot - Save Corruption Test
|
||||
func test_corrupted_save_handling():
|
||||
# Create corrupted save file
|
||||
var file = FileAccess.open("user://save_corrupt.dat", FileAccess.WRITE)
|
||||
file.store_string("CORRUPTED_GARBAGE_DATA")
|
||||
file.close()
|
||||
|
||||
# Attempt to load
|
||||
var result = SaveManager.load("save_corrupt")
|
||||
|
||||
# Should handle gracefully
|
||||
assert_null(result, "Should return null for corrupted save")
|
||||
assert_false(OS.has_feature("crashed"), "Should not crash")
|
||||
|
||||
# Should show user message
|
||||
var message_shown = ErrorDisplay.current_message != ""
|
||||
assert_true(message_shown, "Should inform user of corruption")
|
||||
```
|
||||
|
||||
## Pre-Submission Checklist
|
||||
|
||||
### General Requirements
|
||||
|
||||
- [ ] Game boots to interactive state within platform time limit
|
||||
- [ ] Controller disconnect pauses game
|
||||
- [ ] User sign-out handled correctly
|
||||
- [ ] Save data validates on load
|
||||
- [ ] No crashes in 8+ hours of automated testing
|
||||
- [ ] Memory usage within platform limits
|
||||
- [ ] Load times meet requirements
|
||||
|
||||
### Platform Services
|
||||
|
||||
- [ ] Achievements/Trophies work correctly
|
||||
- [ ] Friends list integration works
|
||||
- [ ] Invite system functions
|
||||
- [ ] Store/DLC integration validated
|
||||
- [ ] Cloud saves sync properly
|
||||
|
||||
### Accessibility (Increasingly Required)
|
||||
|
||||
- [ ] Text size options
|
||||
- [ ] Colorblind modes
|
||||
- [ ] Subtitle options
|
||||
- [ ] Controller remapping
|
||||
- [ ] Screen reader support (where applicable)
|
||||
|
||||
### Content Compliance
|
||||
|
||||
- [ ] Age rating displayed correctly
|
||||
- [ ] Parental controls respected
|
||||
- [ ] No prohibited content
|
||||
- [ ] Required legal text present
|
||||
|
||||
## Common Certification Failures
|
||||
|
||||
| Issue | Platform | Fix |
|
||||
| --------------------- | ------------ | ----------------------------------- |
|
||||
| Home button delay | All consoles | Respond within required time |
|
||||
| Controller timeout | PlayStation | Handle reactivation properly |
|
||||
| Save on suspend | PlayStation | Don't save during suspend |
|
||||
| User context loss | Xbox | Track active user correctly |
|
||||
| Joy-Con drift | Switch | Proper deadzone handling |
|
||||
| Background memory | Mobile | Release resources when backgrounded |
|
||||
| Crash on corrupt data | All | Validate all loaded data |
|
||||
|
||||
## Testing Matrix
|
||||
|
||||
### Build Configurations to Test
|
||||
|
||||
| Configuration | Scenarios |
|
||||
| --------------- | ----------------------- |
|
||||
| First boot | No save data exists |
|
||||
| Return user | Save data present |
|
||||
| Upgrade path | Previous version save |
|
||||
| Fresh install | After uninstall |
|
||||
| Low storage | Minimum space available |
|
||||
| Network offline | No connectivity |
|
||||
|
||||
### Hardware Variants
|
||||
|
||||
| Platform | Variants to Test |
|
||||
| ----------- | ------------------------------- |
|
||||
| PlayStation | PS4, PS4 Pro, PS5 |
|
||||
| Xbox | One, One X, Series S, Series X |
|
||||
| Switch | Docked, Handheld, Lite |
|
||||
| PC | Min spec, recommended, high-end |
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Read platform requirements document thoroughly
|
||||
- Test on actual hardware, not just dev kits
|
||||
- Automate certification test scenarios
|
||||
- Submit with extra time for re-submission
|
||||
- Document all edge case handling
|
||||
- Test with real user accounts
|
||||
|
||||
### DON'T
|
||||
|
||||
- Assume debug builds behave like retail
|
||||
- Skip testing on oldest supported hardware
|
||||
- Ignore platform-specific features
|
||||
- Wait until last minute to test certification items
|
||||
- Use placeholder content in submission build
|
||||
- Skip testing with real platform services
|
||||
|
|
@ -1,228 +0,0 @@
|
|||
# Compatibility Testing for Games
|
||||
|
||||
## Overview
|
||||
|
||||
Compatibility testing ensures your game works correctly across different hardware, operating systems, and configurations that players use.
|
||||
|
||||
## Types of Compatibility Testing
|
||||
|
||||
### Hardware Compatibility
|
||||
|
||||
- Graphics cards (NVIDIA, AMD, Intel)
|
||||
- CPUs (Intel, AMD, Apple Silicon)
|
||||
- Memory configurations
|
||||
- Storage types (HDD, SSD, NVMe)
|
||||
- Input devices (controllers, keyboards, mice)
|
||||
|
||||
### Software Compatibility
|
||||
|
||||
- Operating system versions
|
||||
- Driver versions
|
||||
- Background software conflicts
|
||||
- Antivirus interference
|
||||
|
||||
### Platform Compatibility
|
||||
|
||||
- Console SKUs (PS5, Xbox Series X|S)
|
||||
- PC storefronts (Steam, Epic, GOG)
|
||||
- Mobile devices (iOS, Android)
|
||||
- Cloud gaming services
|
||||
|
||||
### Configuration Compatibility
|
||||
|
||||
- Graphics settings combinations
|
||||
- Resolution and aspect ratios
|
||||
- Refresh rates (60Hz, 144Hz, etc.)
|
||||
- HDR and color profiles
|
||||
|
||||
## Testing Matrix
|
||||
|
||||
### Minimum Hardware Matrix
|
||||
|
||||
| Component | Budget | Mid-Range | High-End |
|
||||
| --------- | -------- | --------- | -------- |
|
||||
| GPU | GTX 1050 | RTX 3060 | RTX 4080 |
|
||||
| CPU | i5-6400 | i7-10700 | i9-13900 |
|
||||
| RAM | 8GB | 16GB | 32GB |
|
||||
| Storage | HDD | SATA SSD | NVMe |
|
||||
|
||||
### OS Matrix
|
||||
|
||||
- Windows 10 (21H2, 22H2)
|
||||
- Windows 11 (22H2, 23H2)
|
||||
- macOS (Ventura, Sonoma)
|
||||
- Linux (Ubuntu LTS, SteamOS)
|
||||
|
||||
### Controller Matrix
|
||||
|
||||
- Xbox Controller (wired, wireless, Elite)
|
||||
- PlayStation DualSense
|
||||
- Nintendo Pro Controller
|
||||
- Generic XInput controllers
|
||||
- Keyboard + Mouse
|
||||
|
||||
## Testing Approach
|
||||
|
||||
### 1. Define Supported Configurations
|
||||
|
||||
- Minimum specifications
|
||||
- Recommended specifications
|
||||
- Officially supported platforms
|
||||
- Known unsupported configurations
|
||||
|
||||
### 2. Create Test Matrix
|
||||
|
||||
- Prioritize common configurations
|
||||
- Include edge cases
|
||||
- Balance coverage vs. effort
|
||||
|
||||
### 3. Execute Systematic Testing
|
||||
|
||||
- Full playthrough on key configs
|
||||
- Spot checks on edge cases
|
||||
- Automated smoke tests where possible
|
||||
|
||||
### 4. Document Issues
|
||||
|
||||
- Repro steps with exact configuration
|
||||
- Severity and frequency
|
||||
- Workarounds if available
|
||||
|
||||
## Common Compatibility Issues
|
||||
|
||||
### Graphics Issues
|
||||
|
||||
| Issue | Cause | Detection |
|
||||
| -------------------- | ---------------------- | -------------------------------- |
|
||||
| Crashes on launch | Driver incompatibility | Test on multiple GPUs |
|
||||
| Rendering artifacts | Shader issues | Visual inspection across configs |
|
||||
| Performance variance | Optimization gaps | Profile on multiple GPUs |
|
||||
| Resolution bugs | Aspect ratio handling | Test non-standard resolutions |
|
||||
|
||||
### Input Issues
|
||||
|
||||
| Issue | Cause | Detection |
|
||||
| ----------------------- | ------------------ | ------------------------------ |
|
||||
| Controller not detected | Missing driver/API | Test all supported controllers |
|
||||
| Wrong button prompts | Platform detection | Swap controllers mid-game |
|
||||
| Stick drift handling | Deadzone issues | Test worn controllers |
|
||||
| Mouse acceleration | Raw input issues | Test at different DPIs |
|
||||
|
||||
### Audio Issues
|
||||
|
||||
| Issue | Cause | Detection |
|
||||
| -------------- | ---------------- | --------------------------- |
|
||||
| No sound | Device selection | Test multiple audio devices |
|
||||
| Crackling | Buffer issues | Test under CPU load |
|
||||
| Wrong channels | Surround setup | Test stereo vs 5.1 vs 7.1 |
|
||||
|
||||
## Platform-Specific Considerations
|
||||
|
||||
### PC
|
||||
|
||||
- **Steam:** Verify Steam Input, Steamworks features
|
||||
- **Epic:** Test EOS features if used
|
||||
- **GOG:** Test offline/DRM-free functionality
|
||||
- **Game Pass:** Test Xbox services integration
|
||||
|
||||
### Console
|
||||
|
||||
- **Certification Requirements:** Study TRCs/XRs early
|
||||
- **SKU Differences:** Test on all variants (S vs X)
|
||||
- **External Storage:** Test on USB drives
|
||||
- **Quick Resume:** Test suspend/resume cycles
|
||||
|
||||
### Mobile
|
||||
|
||||
- **Device Fragmentation:** Test across screen sizes
|
||||
- **OS Versions:** Test min supported to latest
|
||||
- **Permissions:** Test permission flows
|
||||
- **App Lifecycle:** Test background/foreground
|
||||
|
||||
## Automated Compatibility Testing
|
||||
|
||||
### Smoke Tests
|
||||
|
||||
```yaml
|
||||
# Run on matrix of configurations
|
||||
compatibility_test:
|
||||
matrix:
|
||||
os: [windows-10, windows-11, ubuntu-22]
|
||||
gpu: [nvidia, amd, intel]
|
||||
script:
|
||||
- launch_game --headless
|
||||
- verify_main_menu_reached
|
||||
- check_no_errors
|
||||
```
|
||||
|
||||
### Screenshot Comparison
|
||||
|
||||
- Capture screenshots on different GPUs
|
||||
- Compare for rendering differences
|
||||
- Flag significant deviations
|
||||
|
||||
### Cloud Testing Services
|
||||
|
||||
- AWS Device Farm
|
||||
- BrowserStack (web games)
|
||||
- LambdaTest
|
||||
- Sauce Labs
|
||||
|
||||
## Compatibility Checklist
|
||||
|
||||
### Pre-Alpha
|
||||
|
||||
- [ ] Minimum specs defined
|
||||
- [ ] Key platforms identified
|
||||
- [ ] Test matrix created
|
||||
- [ ] Test hardware acquired/rented
|
||||
|
||||
### Alpha
|
||||
|
||||
- [ ] Full playthrough on min spec
|
||||
- [ ] Controller support verified
|
||||
- [ ] Major graphics issues found
|
||||
- [ ] Platform SDK integrated
|
||||
|
||||
### Beta
|
||||
|
||||
- [ ] All matrix configurations tested
|
||||
- [ ] Edge cases explored
|
||||
- [ ] Certification pre-check done
|
||||
- [ ] Store page requirements met
|
||||
|
||||
### Release
|
||||
|
||||
- [ ] Final certification passed
|
||||
- [ ] Known issues documented
|
||||
- [ ] Workarounds communicated
|
||||
- [ ] Support matrix published
|
||||
|
||||
## Documenting Compatibility
|
||||
|
||||
### System Requirements
|
||||
|
||||
```
|
||||
MINIMUM:
|
||||
- OS: Windows 10 64-bit
|
||||
- Processor: Intel Core i5-6400 or AMD equivalent
|
||||
- Memory: 8 GB RAM
|
||||
- Graphics: NVIDIA GTX 1050 or AMD RX 560
|
||||
- Storage: 50 GB available space
|
||||
|
||||
RECOMMENDED:
|
||||
- OS: Windows 11 64-bit
|
||||
- Processor: Intel Core i7-10700 or AMD equivalent
|
||||
- Memory: 16 GB RAM
|
||||
- Graphics: NVIDIA RTX 3060 or AMD RX 6700 XT
|
||||
- Storage: 50 GB SSD
|
||||
```
|
||||
|
||||
### Known Issues
|
||||
|
||||
Maintain a public-facing list of known compatibility issues with:
|
||||
|
||||
- Affected configurations
|
||||
- Symptoms
|
||||
- Workarounds
|
||||
- Fix status
|
||||
|
|
@ -1,376 +0,0 @@
|
|||
# Godot GUT Testing Guide
|
||||
|
||||
## Overview
|
||||
|
||||
GUT (Godot Unit Test) is the standard unit testing framework for Godot. It provides a full-featured testing framework with assertions, mocking, and CI integration.
|
||||
|
||||
## Installation
|
||||
|
||||
### Via Asset Library
|
||||
|
||||
1. Open AssetLib in Godot
|
||||
2. Search for "GUT"
|
||||
3. Download and install
|
||||
4. Enable the plugin in Project Settings
|
||||
|
||||
### Via Git Submodule
|
||||
|
||||
```bash
|
||||
git submodule add https://github.com/bitwes/Gut.git addons/gut
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
project/
|
||||
├── addons/
|
||||
│ └── gut/
|
||||
├── src/
|
||||
│ ├── player/
|
||||
│ │ └── player.gd
|
||||
│ └── combat/
|
||||
│ └── damage_calculator.gd
|
||||
└── tests/
|
||||
├── unit/
|
||||
│ └── test_damage_calculator.gd
|
||||
└── integration/
|
||||
└── test_player_combat.gd
|
||||
```
|
||||
|
||||
## Basic Test Structure
|
||||
|
||||
### Simple Test Class
|
||||
|
||||
```gdscript
|
||||
# tests/unit/test_damage_calculator.gd
|
||||
extends GutTest
|
||||
|
||||
var calculator: DamageCalculator
|
||||
|
||||
func before_each():
|
||||
calculator = DamageCalculator.new()
|
||||
|
||||
func after_each():
|
||||
calculator.free()
|
||||
|
||||
func test_calculate_base_damage():
|
||||
var result = calculator.calculate(100.0, 1.0)
|
||||
assert_eq(result, 100.0, "Base damage should equal input")
|
||||
|
||||
func test_calculate_critical_hit():
|
||||
var result = calculator.calculate(100.0, 2.0)
|
||||
assert_eq(result, 200.0, "Critical hit should double damage")
|
||||
|
||||
func test_calculate_with_zero_multiplier():
|
||||
var result = calculator.calculate(100.0, 0.0)
|
||||
assert_eq(result, 0.0, "Zero multiplier should result in zero damage")
|
||||
```
|
||||
|
||||
### Parameterized Tests
|
||||
|
||||
```gdscript
|
||||
func test_damage_scenarios():
|
||||
var scenarios = [
|
||||
{"base": 100.0, "mult": 1.0, "expected": 100.0},
|
||||
{"base": 100.0, "mult": 2.0, "expected": 200.0},
|
||||
{"base": 50.0, "mult": 1.5, "expected": 75.0},
|
||||
{"base": 0.0, "mult": 2.0, "expected": 0.0},
|
||||
]
|
||||
|
||||
for scenario in scenarios:
|
||||
var result = calculator.calculate(scenario.base, scenario.mult)
|
||||
assert_eq(
|
||||
result,
|
||||
scenario.expected,
|
||||
"Base %s * %s should equal %s" % [
|
||||
scenario.base, scenario.mult, scenario.expected
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
## Testing Nodes
|
||||
|
||||
### Scene Testing
|
||||
|
||||
```gdscript
|
||||
# tests/integration/test_player.gd
|
||||
extends GutTest
|
||||
|
||||
var player: Player
|
||||
var player_scene = preload("res://src/player/player.tscn")
|
||||
|
||||
func before_each():
|
||||
player = player_scene.instantiate()
|
||||
add_child(player)
|
||||
|
||||
func after_each():
|
||||
player.queue_free()
|
||||
|
||||
func test_player_initial_health():
|
||||
assert_eq(player.health, 100, "Player should start with 100 health")
|
||||
|
||||
func test_player_takes_damage():
|
||||
player.take_damage(30)
|
||||
assert_eq(player.health, 70, "Health should be reduced by damage")
|
||||
|
||||
func test_player_dies_at_zero_health():
|
||||
player.take_damage(100)
|
||||
assert_true(player.is_dead, "Player should be dead at 0 health")
|
||||
```
|
||||
|
||||
### Testing with Signals
|
||||
|
||||
```gdscript
|
||||
func test_damage_emits_signal():
|
||||
watch_signals(player)
|
||||
|
||||
player.take_damage(10)
|
||||
|
||||
assert_signal_emitted(player, "health_changed")
|
||||
assert_signal_emit_count(player, "health_changed", 1)
|
||||
|
||||
func test_death_emits_signal():
|
||||
watch_signals(player)
|
||||
|
||||
player.take_damage(100)
|
||||
|
||||
assert_signal_emitted(player, "died")
|
||||
```
|
||||
|
||||
### Testing with Await
|
||||
|
||||
```gdscript
|
||||
func test_attack_cooldown():
|
||||
player.attack()
|
||||
assert_true(player.is_attacking)
|
||||
|
||||
# Wait for cooldown
|
||||
await get_tree().create_timer(player.attack_cooldown).timeout
|
||||
|
||||
assert_false(player.is_attacking)
|
||||
assert_true(player.can_attack)
|
||||
```
|
||||
|
||||
## Mocking and Doubles
|
||||
|
||||
### Creating Doubles
|
||||
|
||||
```gdscript
|
||||
func test_enemy_uses_pathfinding():
|
||||
var mock_pathfinding = double(Pathfinding).new()
|
||||
stub(mock_pathfinding, "find_path").to_return([Vector2(0, 0), Vector2(10, 10)])
|
||||
|
||||
var enemy = Enemy.new()
|
||||
enemy.pathfinding = mock_pathfinding
|
||||
|
||||
enemy.move_to(Vector2(10, 10))
|
||||
|
||||
assert_called(mock_pathfinding, "find_path")
|
||||
```
|
||||
|
||||
### Partial Doubles
|
||||
|
||||
```gdscript
|
||||
func test_player_inventory():
|
||||
var player_double = partial_double(Player).new()
|
||||
stub(player_double, "save_to_disk").to_do_nothing()
|
||||
|
||||
player_double.add_item("sword")
|
||||
|
||||
assert_eq(player_double.inventory.size(), 1)
|
||||
assert_called(player_double, "save_to_disk")
|
||||
```
|
||||
|
||||
## Physics Testing
|
||||
|
||||
### Testing Collision
|
||||
|
||||
```gdscript
|
||||
func test_projectile_hits_enemy():
|
||||
var projectile = Projectile.new()
|
||||
var enemy = Enemy.new()
|
||||
|
||||
add_child(projectile)
|
||||
add_child(enemy)
|
||||
|
||||
projectile.global_position = Vector2(0, 0)
|
||||
enemy.global_position = Vector2(100, 0)
|
||||
|
||||
projectile.velocity = Vector2(200, 0)
|
||||
|
||||
# Simulate physics frames
|
||||
for i in range(60):
|
||||
await get_tree().physics_frame
|
||||
|
||||
assert_true(enemy.was_hit, "Enemy should be hit by projectile")
|
||||
|
||||
projectile.queue_free()
|
||||
enemy.queue_free()
|
||||
```
|
||||
|
||||
### Testing Area2D
|
||||
|
||||
```gdscript
|
||||
func test_pickup_collected():
|
||||
var pickup = Pickup.new()
|
||||
var player = player_scene.instantiate()
|
||||
|
||||
add_child(pickup)
|
||||
add_child(player)
|
||||
|
||||
pickup.global_position = Vector2(50, 50)
|
||||
player.global_position = Vector2(50, 50)
|
||||
|
||||
# Wait for physics to process overlap
|
||||
await get_tree().physics_frame
|
||||
await get_tree().physics_frame
|
||||
|
||||
assert_true(pickup.is_queued_for_deletion(), "Pickup should be collected")
|
||||
|
||||
player.queue_free()
|
||||
```
|
||||
|
||||
## Input Testing
|
||||
|
||||
### Simulating Input
|
||||
|
||||
```gdscript
|
||||
func test_jump_on_input():
|
||||
var input_event = InputEventKey.new()
|
||||
input_event.keycode = KEY_SPACE
|
||||
input_event.pressed = true
|
||||
|
||||
Input.parse_input_event(input_event)
|
||||
await get_tree().process_frame
|
||||
|
||||
player._unhandled_input(input_event)
|
||||
|
||||
assert_true(player.is_jumping, "Player should jump on space press")
|
||||
```
|
||||
|
||||
### Testing Input Actions
|
||||
|
||||
```gdscript
|
||||
func test_attack_action():
|
||||
# Simulate action press
|
||||
Input.action_press("attack")
|
||||
await get_tree().process_frame
|
||||
|
||||
player._process(0.016)
|
||||
|
||||
assert_true(player.is_attacking)
|
||||
|
||||
Input.action_release("attack")
|
||||
```
|
||||
|
||||
## Resource Testing
|
||||
|
||||
### Testing Custom Resources
|
||||
|
||||
```gdscript
|
||||
func test_weapon_stats_resource():
|
||||
var weapon = WeaponStats.new()
|
||||
weapon.base_damage = 10.0
|
||||
weapon.attack_speed = 2.0
|
||||
|
||||
assert_eq(weapon.dps, 20.0, "DPS should be damage * speed")
|
||||
|
||||
func test_save_load_resource():
|
||||
var original = PlayerData.new()
|
||||
original.level = 5
|
||||
original.gold = 1000
|
||||
|
||||
ResourceSaver.save(original, "user://test_save.tres")
|
||||
var loaded = ResourceLoader.load("user://test_save.tres")
|
||||
|
||||
assert_eq(loaded.level, 5)
|
||||
assert_eq(loaded.gold, 1000)
|
||||
|
||||
DirAccess.remove_absolute("user://test_save.tres")
|
||||
```
|
||||
|
||||
## GUT Configuration
|
||||
|
||||
### gut_config.json
|
||||
|
||||
```json
|
||||
{
|
||||
"dirs": ["res://tests/"],
|
||||
"include_subdirs": true,
|
||||
"prefix": "test_",
|
||||
"suffix": ".gd",
|
||||
"should_exit": true,
|
||||
"should_exit_on_success": true,
|
||||
"log_level": 1,
|
||||
"junit_xml_file": "results.xml",
|
||||
"font_size": 16
|
||||
}
|
||||
```
|
||||
|
||||
## CI Integration
|
||||
|
||||
### Command Line Execution
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
godot --headless -s addons/gut/gut_cmdln.gd
|
||||
|
||||
# Run specific tests
|
||||
godot --headless -s addons/gut/gut_cmdln.gd \
|
||||
-gdir=res://tests/unit \
|
||||
-gprefix=test_
|
||||
|
||||
# With JUnit output
|
||||
godot --headless -s addons/gut/gut_cmdln.gd \
|
||||
-gjunit_xml_file=results.xml
|
||||
```
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
```yaml
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: barichello/godot-ci:4.2
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Run Tests
|
||||
run: |
|
||||
godot --headless -s addons/gut/gut_cmdln.gd \
|
||||
-gjunit_xml_file=results.xml
|
||||
|
||||
- name: Publish Results
|
||||
uses: mikepenz/action-junit-report@v4
|
||||
with:
|
||||
report_paths: results.xml
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Use `before_each`/`after_each` for setup/teardown
|
||||
- Free nodes after tests to prevent leaks
|
||||
- Use meaningful assertion messages
|
||||
- Group related tests in the same file
|
||||
- Use `watch_signals` for signal testing
|
||||
- Await physics frames when testing physics
|
||||
|
||||
### DON'T
|
||||
|
||||
- Don't test Godot's built-in functionality
|
||||
- Don't rely on execution order between test files
|
||||
- Don't leave orphan nodes
|
||||
- Don't use `yield` (use `await` in Godot 4)
|
||||
- Don't test private methods directly
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Cause | Fix |
|
||||
| -------------------- | ------------------ | ------------------------------------ |
|
||||
| Tests not found | Wrong prefix/path | Check gut_config.json |
|
||||
| Orphan nodes warning | Missing cleanup | Add `queue_free()` in `after_each` |
|
||||
| Signal not detected | Signal not watched | Call `watch_signals()` before action |
|
||||
| Physics not working | Missing frames | Await `physics_frame` |
|
||||
| Flaky tests | Timing issues | Use proper await/signals |
|
||||
|
|
@ -1,315 +0,0 @@
|
|||
# Input Testing Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Input testing validates that all supported input devices work correctly across platforms. Poor input handling frustrates players instantly—responsive, accurate input is foundational to game feel.
|
||||
|
||||
## Input Categories
|
||||
|
||||
### Device Types
|
||||
|
||||
| Device | Platforms | Key Concerns |
|
||||
| ----------------- | -------------- | ----------------------------------- |
|
||||
| Keyboard + Mouse | PC | Key conflicts, DPI sensitivity |
|
||||
| Gamepad (Xbox/PS) | PC, Console | Deadzone, vibration, button prompts |
|
||||
| Touch | Mobile, Switch | Multi-touch, gesture recognition |
|
||||
| Motion Controls | Switch, VR | Calibration, drift, fatigue |
|
||||
| Specialty | Various | Flight sticks, wheels, fight sticks |
|
||||
|
||||
### Input Characteristics
|
||||
|
||||
| Characteristic | Description | Test Focus |
|
||||
| -------------- | ---------------------------- | -------------------------------- |
|
||||
| Responsiveness | Input-to-action delay | Should feel instant (< 100ms) |
|
||||
| Accuracy | Input maps to correct action | No ghost inputs or missed inputs |
|
||||
| Consistency | Same input = same result | Deterministic behavior |
|
||||
| Accessibility | Alternative input support | Remapping, assist options |
|
||||
|
||||
## Test Scenarios
|
||||
|
||||
### Keyboard and Mouse
|
||||
|
||||
```
|
||||
SCENARIO: All Keybinds Functional
|
||||
GIVEN default keyboard bindings
|
||||
WHEN each bound key is pressed
|
||||
THEN corresponding action triggers
|
||||
AND no key conflicts exist
|
||||
|
||||
SCENARIO: Key Remapping
|
||||
GIVEN player remaps "Jump" from Space to F
|
||||
WHEN F is pressed
|
||||
THEN jump action triggers
|
||||
AND Space no longer triggers jump
|
||||
AND remapping persists after restart
|
||||
|
||||
SCENARIO: Mouse Sensitivity
|
||||
GIVEN sensitivity set to 5 (mid-range)
|
||||
WHEN mouse moves 10cm
|
||||
THEN camera rotation matches expected degrees
|
||||
AND movement feels consistent at different frame rates
|
||||
|
||||
SCENARIO: Mouse Button Support
|
||||
GIVEN mouse with 5+ buttons
|
||||
WHEN side buttons are pressed
|
||||
THEN they can be bound to actions
|
||||
AND they function correctly in gameplay
|
||||
```
|
||||
|
||||
### Gamepad
|
||||
|
||||
```
|
||||
SCENARIO: Analog Stick Deadzone
|
||||
GIVEN controller with slight stick drift
|
||||
WHEN stick is in neutral position
|
||||
THEN no movement occurs (deadzone filters drift)
|
||||
AND intentional small movements still register
|
||||
|
||||
SCENARIO: Trigger Pressure
|
||||
GIVEN analog triggers
|
||||
WHEN trigger is partially pressed
|
||||
THEN partial values are read (e.g., 0.5 for half-press)
|
||||
AND full press reaches 1.0
|
||||
|
||||
SCENARIO: Controller Hot-Swap
|
||||
GIVEN game running with keyboard
|
||||
WHEN gamepad is connected
|
||||
THEN input prompts switch to gamepad icons
|
||||
AND gamepad input works immediately
|
||||
AND keyboard still works if used
|
||||
|
||||
SCENARIO: Vibration Feedback
|
||||
GIVEN rumble-enabled controller
|
||||
WHEN damage is taken
|
||||
THEN controller vibrates appropriately
|
||||
AND vibration intensity matches damage severity
|
||||
```
|
||||
|
||||
### Touch Input
|
||||
|
||||
```
|
||||
SCENARIO: Multi-Touch Accuracy
|
||||
GIVEN virtual joystick and buttons
|
||||
WHEN left thumb on joystick AND right thumb on button
|
||||
THEN both inputs register simultaneously
|
||||
AND no interference between touch points
|
||||
|
||||
SCENARIO: Gesture Recognition
|
||||
GIVEN swipe-to-attack mechanic
|
||||
WHEN player swipes right
|
||||
THEN attack direction matches swipe
|
||||
AND swipe is distinguished from tap
|
||||
|
||||
SCENARIO: Touch Target Size
|
||||
GIVEN minimum touch target of 44x44 points
|
||||
WHEN buttons are placed
|
||||
THEN all interactive elements meet minimum size
|
||||
AND elements have adequate spacing
|
||||
```
|
||||
|
||||
## Platform-Specific Testing
|
||||
|
||||
### PC
|
||||
|
||||
- Multiple keyboard layouts (QWERTY, AZERTY, QWERTZ)
|
||||
- Different mouse DPI settings (400-3200+)
|
||||
- Multiple monitors (cursor confinement)
|
||||
- Background application conflicts
|
||||
- Steam Input API integration
|
||||
|
||||
### Console
|
||||
|
||||
| Platform | Specific Tests |
|
||||
| ----------- | ------------------------------------------ |
|
||||
| PlayStation | Touchpad, adaptive triggers, haptics |
|
||||
| Xbox | Impulse triggers, Elite controller paddles |
|
||||
| Switch | Joy-Con detachment, gyro, HD rumble |
|
||||
|
||||
### Mobile
|
||||
|
||||
- Different screen sizes and aspect ratios
|
||||
- Notch/cutout avoidance
|
||||
- External controller support
|
||||
- Apple MFi / Android gamepad compatibility
|
||||
|
||||
## Automated Test Examples
|
||||
|
||||
### Unity
|
||||
|
||||
```csharp
|
||||
using UnityEngine.InputSystem;
|
||||
|
||||
[UnityTest]
|
||||
public IEnumerator Movement_WithGamepad_RespondsToStick()
|
||||
{
|
||||
var gamepad = InputSystem.AddDevice<Gamepad>();
|
||||
|
||||
yield return null;
|
||||
|
||||
// Simulate stick input
|
||||
Set(gamepad.leftStick, new Vector2(1, 0));
|
||||
yield return new WaitForSeconds(0.1f);
|
||||
|
||||
Assert.Greater(player.transform.position.x, 0f,
|
||||
"Player should move right");
|
||||
|
||||
InputSystem.RemoveDevice(gamepad);
|
||||
}
|
||||
|
||||
[UnityTest]
|
||||
public IEnumerator InputLatency_UnderLoad_StaysAcceptable()
|
||||
{
|
||||
float inputTime = Time.realtimeSinceStartup;
|
||||
bool actionTriggered = false;
|
||||
|
||||
player.OnJump += () => {
|
||||
float latency = (Time.realtimeSinceStartup - inputTime) * 1000;
|
||||
Assert.Less(latency, 100f, "Input latency should be under 100ms");
|
||||
actionTriggered = true;
|
||||
};
|
||||
|
||||
var keyboard = InputSystem.AddDevice<Keyboard>();
|
||||
Press(keyboard.spaceKey);
|
||||
|
||||
yield return new WaitForSeconds(0.2f);
|
||||
|
||||
Assert.IsTrue(actionTriggered, "Jump should have triggered");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Deadzone_FiltersSmallInputs()
|
||||
{
|
||||
var settings = new InputSettings { stickDeadzone = 0.2f };
|
||||
|
||||
// Input below deadzone
|
||||
var filtered = InputProcessor.ApplyDeadzone(new Vector2(0.1f, 0.1f), settings);
|
||||
Assert.AreEqual(Vector2.zero, filtered);
|
||||
|
||||
// Input above deadzone
|
||||
filtered = InputProcessor.ApplyDeadzone(new Vector2(0.5f, 0.5f), settings);
|
||||
Assert.AreNotEqual(Vector2.zero, filtered);
|
||||
}
|
||||
```
|
||||
|
||||
### Unreal
|
||||
|
||||
```cpp
|
||||
bool FInputTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
// Test gamepad input mapping
|
||||
APlayerController* PC = GetWorld()->GetFirstPlayerController();
|
||||
|
||||
// Simulate gamepad stick input
|
||||
FInputKeyParams Params;
|
||||
Params.Key = EKeys::Gamepad_LeftX;
|
||||
Params.Delta = FVector(1.0f, 0, 0);
|
||||
PC->InputKey(Params);
|
||||
|
||||
// Verify movement
|
||||
APawn* Pawn = PC->GetPawn();
|
||||
FVector Velocity = Pawn->GetVelocity();
|
||||
|
||||
TestTrue("Pawn should be moving", Velocity.SizeSquared() > 0);
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Godot
|
||||
|
||||
```gdscript
|
||||
func test_input_action_mapping():
|
||||
# Verify action exists
|
||||
assert_true(InputMap.has_action("jump"))
|
||||
|
||||
# Simulate input
|
||||
var event = InputEventKey.new()
|
||||
event.keycode = KEY_SPACE
|
||||
event.pressed = true
|
||||
|
||||
Input.parse_input_event(event)
|
||||
await get_tree().process_frame
|
||||
|
||||
assert_true(Input.is_action_just_pressed("jump"))
|
||||
|
||||
func test_gamepad_deadzone():
|
||||
var input = Vector2(0.15, 0.1)
|
||||
var deadzone = 0.2
|
||||
|
||||
var processed = input_processor.apply_deadzone(input, deadzone)
|
||||
|
||||
assert_eq(processed, Vector2.ZERO, "Small input should be filtered")
|
||||
|
||||
func test_controller_hotswap():
|
||||
# Simulate controller connect
|
||||
Input.joy_connection_changed(0, true)
|
||||
await get_tree().process_frame
|
||||
|
||||
var prompt_icon = ui.get_action_prompt("jump")
|
||||
|
||||
assert_true(prompt_icon.texture.resource_path.contains("gamepad"),
|
||||
"Should show gamepad prompts after controller connect")
|
||||
```
|
||||
|
||||
## Accessibility Testing
|
||||
|
||||
### Requirements Checklist
|
||||
|
||||
- [ ] Full keyboard navigation (no mouse required)
|
||||
- [ ] Remappable controls for all actions
|
||||
- [ ] Button hold alternatives to rapid press
|
||||
- [ ] Toggle options for hold actions
|
||||
- [ ] One-handed control schemes
|
||||
- [ ] Colorblind-friendly UI indicators
|
||||
- [ ] Screen reader support for menus
|
||||
|
||||
### Accessibility Test Scenarios
|
||||
|
||||
```
|
||||
SCENARIO: Keyboard-Only Navigation
|
||||
GIVEN mouse is disconnected
|
||||
WHEN navigating through all menus
|
||||
THEN all menu items are reachable via keyboard
|
||||
AND focus indicators are clearly visible
|
||||
|
||||
SCENARIO: Button Hold Toggle
|
||||
GIVEN "sprint requires hold" is toggled OFF
|
||||
WHEN sprint button is tapped once
|
||||
THEN sprint activates
|
||||
AND sprint stays active until tapped again
|
||||
|
||||
SCENARIO: Reduced Button Mashing
|
||||
GIVEN QTE assist mode enabled
|
||||
WHEN QTE sequence appears
|
||||
THEN single press advances sequence
|
||||
AND no rapid input required
|
||||
```
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
| Metric | Target | Maximum Acceptable |
|
||||
| ----------------------- | --------------- | ------------------ |
|
||||
| Input-to-render latency | < 50ms | 100ms |
|
||||
| Polling rate match | 1:1 with device | No input loss |
|
||||
| Deadzone processing | < 1ms | 5ms |
|
||||
| Rebind save/load | < 100ms | 500ms |
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Test with actual hardware, not just simulated input
|
||||
- Support simultaneous keyboard + gamepad
|
||||
- Provide sensible default deadzones
|
||||
- Show device-appropriate button prompts
|
||||
- Allow complete control remapping
|
||||
- Test at different frame rates
|
||||
|
||||
### DON'T
|
||||
|
||||
- Assume controller layout (Xbox vs PlayStation)
|
||||
- Hard-code input mappings
|
||||
- Ignore analog input precision
|
||||
- Skip accessibility considerations
|
||||
- Forget about input during loading/cutscenes
|
||||
- Neglect testing with worn/drifting controllers
|
||||
|
|
@ -1,304 +0,0 @@
|
|||
# Localization Testing Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Localization testing ensures games work correctly across languages, regions, and cultures. Beyond translation, it validates text display, cultural appropriateness, and regional compliance.
|
||||
|
||||
## Test Categories
|
||||
|
||||
### Linguistic Testing
|
||||
|
||||
| Category | Focus | Examples |
|
||||
| -------------------- | ----------------------- | ------------------------------ |
|
||||
| Translation accuracy | Meaning preserved | Idioms, game terminology |
|
||||
| Grammar/spelling | Language correctness | Verb tense, punctuation |
|
||||
| Consistency | Same terms throughout | "Health" vs "HP" vs "Life" |
|
||||
| Context | Meaning in game context | Item names, skill descriptions |
|
||||
|
||||
### Functional Testing
|
||||
|
||||
| Category | Focus | Examples |
|
||||
| -------------- | ----------------------- | --------------------------- |
|
||||
| Text display | Fits in UI | Button labels, dialog boxes |
|
||||
| Font support | Characters render | CJK, Cyrillic, Arabic |
|
||||
| Text expansion | Longer translations | German is ~30% longer |
|
||||
| RTL support | Right-to-left languages | Arabic, Hebrew layouts |
|
||||
|
||||
### Cultural Testing
|
||||
|
||||
| Category | Focus | Examples |
|
||||
| -------------------- | ------------------ | ------------------------- |
|
||||
| Cultural sensitivity | Offensive content | Gestures, symbols, colors |
|
||||
| Regional compliance | Legal requirements | Ratings, gambling laws |
|
||||
| Date/time formats | Local conventions | DD/MM/YYYY vs MM/DD/YYYY |
|
||||
| Number formats | Decimal separators | 1,000.00 vs 1.000,00 |
|
||||
|
||||
## Test Scenarios
|
||||
|
||||
### Text Display
|
||||
|
||||
```
|
||||
SCENARIO: Text Fits UI Elements
|
||||
GIVEN all localized strings
|
||||
WHEN displayed in target language
|
||||
THEN text fits within UI boundaries
|
||||
AND no truncation or overflow occurs
|
||||
AND text remains readable
|
||||
|
||||
SCENARIO: Dynamic Text Insertion
|
||||
GIVEN template "Player {name} scored {points} points"
|
||||
WHEN name="Alexander" and points=1000
|
||||
THEN German: "Spieler Alexander hat 1.000 Punkte erzielt"
|
||||
AND text fits UI element
|
||||
AND variables are correctly formatted for locale
|
||||
|
||||
SCENARIO: Plural Forms
|
||||
GIVEN English "1 coin" / "5 coins"
|
||||
WHEN displaying in Polish (4 plural forms)
|
||||
THEN correct plural form is used
|
||||
AND all plural forms are translated
|
||||
```
|
||||
|
||||
### Character Support
|
||||
|
||||
```
|
||||
SCENARIO: CJK Character Rendering
|
||||
GIVEN Japanese localization
|
||||
WHEN displaying text with kanji/hiragana/katakana
|
||||
THEN all characters render correctly
|
||||
AND no missing glyphs (tofu boxes)
|
||||
AND line breaks respect CJK rules
|
||||
|
||||
SCENARIO: Special Characters
|
||||
GIVEN text with accented characters (é, ñ, ü)
|
||||
WHEN displayed in-game
|
||||
THEN all characters render correctly
|
||||
AND sorting works correctly
|
||||
|
||||
SCENARIO: User-Generated Content
|
||||
GIVEN player can name character
|
||||
WHEN name includes non-Latin characters
|
||||
THEN name displays correctly
|
||||
AND name saves/loads correctly
|
||||
AND name appears correctly to other players
|
||||
```
|
||||
|
||||
### Layout and Direction
|
||||
|
||||
```
|
||||
SCENARIO: Right-to-Left Layout
|
||||
GIVEN Arabic localization
|
||||
WHEN viewing UI
|
||||
THEN text reads right-to-left
|
||||
AND UI elements mirror appropriately
|
||||
AND numbers remain left-to-right
|
||||
AND mixed content (Arabic + English) displays correctly
|
||||
|
||||
SCENARIO: Text Expansion Accommodation
|
||||
GIVEN English UI "OK" / "Cancel" buttons
|
||||
WHEN localized to German "OK" / "Abbrechen"
|
||||
THEN button expands or text size adjusts
|
||||
AND button remains clickable
|
||||
AND layout doesn't break
|
||||
```
|
||||
|
||||
## Locale-Specific Formatting
|
||||
|
||||
### Date and Time
|
||||
|
||||
| Locale | Date Format | Time Format |
|
||||
| ------ | -------------- | ----------- |
|
||||
| en-US | 12/25/2024 | 3:30 PM |
|
||||
| en-GB | 25/12/2024 | 15:30 |
|
||||
| de-DE | 25.12.2024 | 15:30 Uhr |
|
||||
| ja-JP | 2024年12月25日 | 15時30分 |
|
||||
|
||||
### Numbers and Currency
|
||||
|
||||
| Locale | Number | Currency |
|
||||
| ------ | -------- | ---------- |
|
||||
| en-US | 1,234.56 | $1,234.56 |
|
||||
| de-DE | 1.234,56 | 1.234,56 € |
|
||||
| fr-FR | 1 234,56 | 1 234,56 € |
|
||||
| ja-JP | 1,234.56 | ¥1,235 |
|
||||
|
||||
## Automated Test Examples
|
||||
|
||||
### Unity
|
||||
|
||||
```csharp
|
||||
using UnityEngine.Localization;
|
||||
|
||||
[Test]
|
||||
public void Localization_AllKeysHaveTranslations([Values("en", "de", "ja", "zh-CN")] string locale)
|
||||
{
|
||||
var stringTable = LocalizationSettings.StringDatabase
|
||||
.GetTable("GameStrings", new Locale(locale));
|
||||
|
||||
foreach (var entry in stringTable)
|
||||
{
|
||||
Assert.IsFalse(string.IsNullOrEmpty(entry.Value.LocalizedValue),
|
||||
$"Missing translation for '{entry.Key}' in {locale}");
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void TextFits_AllUIElements()
|
||||
{
|
||||
var languages = new[] { "en", "de", "fr", "ja" };
|
||||
|
||||
foreach (var lang in languages)
|
||||
{
|
||||
LocalizationSettings.SelectedLocale = new Locale(lang);
|
||||
|
||||
foreach (var textElement in FindObjectsOfType<LocalizedText>())
|
||||
{
|
||||
var rectTransform = textElement.GetComponent<RectTransform>();
|
||||
var textComponent = textElement.GetComponent<Text>();
|
||||
|
||||
Assert.LessOrEqual(
|
||||
textComponent.preferredWidth,
|
||||
rectTransform.rect.width,
|
||||
$"Text overflows in {lang}: {textElement.name}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
[TestCase("en", 1, "1 coin")]
|
||||
[TestCase("en", 5, "5 coins")]
|
||||
[TestCase("ru", 1, "1 монета")]
|
||||
[TestCase("ru", 2, "2 монеты")]
|
||||
[TestCase("ru", 5, "5 монет")]
|
||||
public void Pluralization_ReturnsCorrectForm(string locale, int count, string expected)
|
||||
{
|
||||
var result = Localization.GetPlural("coin", count, locale);
|
||||
Assert.AreEqual(expected, result);
|
||||
}
|
||||
```
|
||||
|
||||
### Unreal
|
||||
|
||||
```cpp
|
||||
bool FLocalizationTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
TArray<FString> Cultures = {"en", "de", "ja", "ko"};
|
||||
|
||||
for (const FString& Culture : Cultures)
|
||||
{
|
||||
FInternationalization::Get().SetCurrentCulture(Culture);
|
||||
|
||||
// Test critical strings exist
|
||||
FText LocalizedText = NSLOCTEXT("Game", "StartButton", "Start");
|
||||
TestFalse(
|
||||
FString::Printf(TEXT("Missing StartButton in %s"), *Culture),
|
||||
LocalizedText.IsEmpty());
|
||||
|
||||
// Test number formatting
|
||||
FText NumberText = FText::AsNumber(1234567);
|
||||
TestTrue(
|
||||
TEXT("Number should be formatted"),
|
||||
NumberText.ToString().Len() > 7); // Has separators
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Godot
|
||||
|
||||
```gdscript
|
||||
func test_all_translations_complete():
|
||||
var locales = ["en", "de", "ja", "es"]
|
||||
var keys = TranslationServer.get_all_keys()
|
||||
|
||||
for locale in locales:
|
||||
TranslationServer.set_locale(locale)
|
||||
for key in keys:
|
||||
var translated = tr(key)
|
||||
assert_ne(translated, key,
|
||||
"Missing translation for '%s' in %s" % [key, locale])
|
||||
|
||||
func test_plural_forms():
|
||||
TranslationServer.set_locale("ru")
|
||||
|
||||
assert_eq(tr_n("coin", "coins", 1), "1 монета")
|
||||
assert_eq(tr_n("coin", "coins", 2), "2 монеты")
|
||||
assert_eq(tr_n("coin", "coins", 5), "5 монет")
|
||||
assert_eq(tr_n("coin", "coins", 21), "21 монета")
|
||||
|
||||
func test_text_fits_buttons():
|
||||
var locales = ["en", "de", "fr"]
|
||||
|
||||
for locale in locales:
|
||||
TranslationServer.set_locale(locale)
|
||||
await get_tree().process_frame # Allow UI update
|
||||
|
||||
for button in get_tree().get_nodes_in_group("localized_buttons"):
|
||||
var label = button.get_node("Label")
|
||||
assert_lt(label.size.x, button.size.x,
|
||||
"Button text overflows in %s: %s" % [locale, button.name])
|
||||
```
|
||||
|
||||
## Visual Verification Checklist
|
||||
|
||||
### Text Display
|
||||
|
||||
- [ ] No truncation in any language
|
||||
- [ ] Consistent font sizing
|
||||
- [ ] Proper line breaks
|
||||
- [ ] No overlapping text
|
||||
|
||||
### UI Layout
|
||||
|
||||
- [ ] Buttons accommodate longer text
|
||||
- [ ] Dialog boxes resize appropriately
|
||||
- [ ] Menu items align correctly
|
||||
- [ ] Scrollbars appear when needed
|
||||
|
||||
### Cultural Elements
|
||||
|
||||
- [ ] Icons are culturally appropriate
|
||||
- [ ] Colors don't have negative connotations
|
||||
- [ ] Gestures are region-appropriate
|
||||
- [ ] No unintended political references
|
||||
|
||||
## Regional Compliance
|
||||
|
||||
### Ratings Requirements
|
||||
|
||||
| Region | Rating Board | Special Requirements |
|
||||
| ------------- | ------------ | ------------------------- |
|
||||
| North America | ESRB | Content descriptors |
|
||||
| Europe | PEGI | Age-appropriate icons |
|
||||
| Japan | CERO | Strict content guidelines |
|
||||
| Germany | USK | Violence restrictions |
|
||||
| China | GRAC | Approval process |
|
||||
|
||||
### Common Regional Issues
|
||||
|
||||
| Issue | Regions Affected | Solution |
|
||||
| ---------------- | ---------------- | ------------------------ |
|
||||
| Blood color | Japan, Germany | Option for green/disable |
|
||||
| Gambling imagery | Many regions | Remove or modify |
|
||||
| Skulls/bones | China | Alternative designs |
|
||||
| Nazi imagery | Germany | Remove entirely |
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Test with native speakers
|
||||
- Plan for text expansion (reserve 30% extra space)
|
||||
- Use placeholder text during development (Lorem ipsum-style)
|
||||
- Support multiple input methods (IME for CJK)
|
||||
- Test all language combinations (UI language + audio language)
|
||||
- Validate string format parameters
|
||||
|
||||
### DON'T
|
||||
|
||||
- Hard-code strings in source code
|
||||
- Assume left-to-right layout
|
||||
- Concatenate translated strings
|
||||
- Use machine translation without review
|
||||
- Forget about date/time/number formatting
|
||||
- Ignore cultural context of images and icons
|
||||
|
|
@ -1,322 +0,0 @@
|
|||
# Multiplayer Testing Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Multiplayer testing validates network code, synchronization, and the player experience under real-world conditions. Network bugs are notoriously hard to reproduce—systematic testing is essential.
|
||||
|
||||
## Test Categories
|
||||
|
||||
### Synchronization Testing
|
||||
|
||||
| Test Type | Description | Priority |
|
||||
| ------------------- | ---------------------------------------- | -------- |
|
||||
| State sync | All clients see consistent game state | P0 |
|
||||
| Position sync | Character positions match across clients | P0 |
|
||||
| Event ordering | Actions occur in correct sequence | P0 |
|
||||
| Conflict resolution | Simultaneous actions handled correctly | P1 |
|
||||
| Late join | New players sync correctly mid-game | P1 |
|
||||
|
||||
### Network Conditions
|
||||
|
||||
| Condition | Simulation Method | Test Focus |
|
||||
| --------------- | ----------------- | ------------------------ |
|
||||
| High latency | 200-500ms delay | Input responsiveness |
|
||||
| Packet loss | 5-20% drop rate | State recovery |
|
||||
| Jitter | Variable delay | Interpolation smoothness |
|
||||
| Bandwidth limit | Throttle to 1Mbps | Data prioritization |
|
||||
| Disconnection | Kill connection | Reconnection handling |
|
||||
|
||||
## Test Scenarios
|
||||
|
||||
### Basic Multiplayer
|
||||
|
||||
```
|
||||
SCENARIO: Player Join/Leave
|
||||
GIVEN host has started multiplayer session
|
||||
WHEN Player 2 joins
|
||||
THEN Player 2 appears in host's game
|
||||
AND Player 1 appears in Player 2's game
|
||||
AND player counts sync across all clients
|
||||
|
||||
SCENARIO: State Synchronization
|
||||
GIVEN 4 players in match
|
||||
WHEN Player 1 picks up item at position (10, 5)
|
||||
THEN item disappears for all players
|
||||
AND Player 1's inventory updates for all players
|
||||
AND no duplicate pickups possible
|
||||
|
||||
SCENARIO: Combat Synchronization
|
||||
GIVEN Player 1 attacks Player 2
|
||||
WHEN attack hits
|
||||
THEN damage is consistent on all clients
|
||||
AND hit effects play for all players
|
||||
AND health updates sync within 100ms
|
||||
```
|
||||
|
||||
### Network Degradation
|
||||
|
||||
```
|
||||
SCENARIO: High Latency Gameplay
|
||||
GIVEN 200ms latency between players
|
||||
WHEN Player 1 moves forward
|
||||
THEN movement is smooth on Player 1's screen
|
||||
AND other players see interpolated movement
|
||||
AND position converges within 500ms
|
||||
|
||||
SCENARIO: Packet Loss Recovery
|
||||
GIVEN 10% packet loss
|
||||
WHEN important game event occurs (goal, kill, etc.)
|
||||
THEN event is eventually delivered
|
||||
AND game state remains consistent
|
||||
AND no duplicate events processed
|
||||
|
||||
SCENARIO: Player Disconnection
|
||||
GIVEN Player 2 disconnects unexpectedly
|
||||
WHEN 5 seconds pass
|
||||
THEN other players are notified
|
||||
AND Player 2's character handles gracefully (despawn/AI takeover)
|
||||
AND game continues without crash
|
||||
```
|
||||
|
||||
### Edge Cases
|
||||
|
||||
```
|
||||
SCENARIO: Simultaneous Actions
|
||||
GIVEN Player 1 and Player 2 grab same item simultaneously
|
||||
WHEN both inputs arrive at server
|
||||
THEN only one player receives item
|
||||
AND other player sees consistent state
|
||||
AND no item duplication
|
||||
|
||||
SCENARIO: Host Migration
|
||||
GIVEN host disconnects
|
||||
WHEN migration begins
|
||||
THEN new host is selected
|
||||
AND game state transfers correctly
|
||||
AND gameplay resumes within 10 seconds
|
||||
|
||||
SCENARIO: Reconnection
|
||||
GIVEN Player 2 disconnects temporarily
|
||||
WHEN Player 2 reconnects within 60 seconds
|
||||
THEN Player 2 rejoins same session
|
||||
AND state is synchronized
|
||||
AND progress is preserved
|
||||
```
|
||||
|
||||
## Network Simulation Tools
|
||||
|
||||
### Unity
|
||||
|
||||
```csharp
|
||||
// Using Unity Transport with Network Simulator
|
||||
using Unity.Netcode;
|
||||
|
||||
public class NetworkSimulator : MonoBehaviour
|
||||
{
|
||||
[SerializeField] private int latencyMs = 100;
|
||||
[SerializeField] private float packetLossPercent = 5f;
|
||||
[SerializeField] private int jitterMs = 20;
|
||||
|
||||
void Start()
|
||||
{
|
||||
var transport = NetworkManager.Singleton.GetComponent<UnityTransport>();
|
||||
var simulator = transport.GetSimulatorParameters();
|
||||
|
||||
simulator.PacketDelayMS = latencyMs;
|
||||
simulator.PacketDropRate = (int)(packetLossPercent * 100);
|
||||
simulator.PacketJitterMS = jitterMs;
|
||||
}
|
||||
}
|
||||
|
||||
// Test
|
||||
[UnityTest]
|
||||
public IEnumerator Position_UnderLatency_ConvergesWithinThreshold()
|
||||
{
|
||||
EnableNetworkSimulation(latencyMs: 200);
|
||||
|
||||
// Move player
|
||||
player1.Move(Vector3.forward * 10);
|
||||
|
||||
yield return new WaitForSeconds(1f);
|
||||
|
||||
// Check other client's view
|
||||
var player1OnClient2 = client2.GetPlayerPosition(player1.Id);
|
||||
var actualPosition = player1.transform.position;
|
||||
|
||||
Assert.Less(Vector3.Distance(player1OnClient2, actualPosition), 0.5f);
|
||||
}
|
||||
```
|
||||
|
||||
### Unreal
|
||||
|
||||
```cpp
|
||||
// Using Network Emulation
|
||||
void UNetworkTestHelper::EnableLatencySimulation(int32 LatencyMs)
|
||||
{
|
||||
if (UNetDriver* NetDriver = GetWorld()->GetNetDriver())
|
||||
{
|
||||
FPacketSimulationSettings Settings;
|
||||
Settings.PktLag = LatencyMs;
|
||||
Settings.PktLagVariance = LatencyMs / 10;
|
||||
Settings.PktLoss = 0;
|
||||
|
||||
NetDriver->SetPacketSimulationSettings(Settings);
|
||||
}
|
||||
}
|
||||
|
||||
// Functional test for sync
|
||||
void AMultiplayerSyncTest::StartTest()
|
||||
{
|
||||
Super::StartTest();
|
||||
|
||||
// Spawn item on server
|
||||
APickupItem* Item = GetWorld()->SpawnActor<APickupItem>(
|
||||
ItemClass, FVector(0, 0, 100));
|
||||
|
||||
// Wait for replication
|
||||
FTimerHandle TimerHandle;
|
||||
GetWorld()->GetTimerManager().SetTimer(TimerHandle, [this, Item]()
|
||||
{
|
||||
// Verify client has item
|
||||
if (VerifyItemExistsOnAllClients(Item))
|
||||
{
|
||||
FinishTest(EFunctionalTestResult::Succeeded, "Item replicated");
|
||||
}
|
||||
else
|
||||
{
|
||||
FinishTest(EFunctionalTestResult::Failed, "Item not found on clients");
|
||||
}
|
||||
}, 2.0f, false);
|
||||
}
|
||||
```
|
||||
|
||||
### Godot
|
||||
|
||||
```gdscript
|
||||
# Network simulation
|
||||
extends Node
|
||||
|
||||
var simulated_latency_ms := 0
|
||||
var packet_loss_percent := 0.0
|
||||
|
||||
func _ready():
|
||||
# Hook into network to simulate conditions
|
||||
multiplayer.peer_packet_received.connect(_on_packet_received)
|
||||
|
||||
func _on_packet_received(id: int, packet: PackedByteArray):
|
||||
if packet_loss_percent > 0 and randf() < packet_loss_percent / 100:
|
||||
return # Drop packet
|
||||
|
||||
if simulated_latency_ms > 0:
|
||||
await get_tree().create_timer(simulated_latency_ms / 1000.0).timeout
|
||||
|
||||
_process_packet(id, packet)
|
||||
|
||||
# Test
|
||||
func test_position_sync_under_latency():
|
||||
NetworkSimulator.simulated_latency_ms = 200
|
||||
|
||||
# Move player on host
|
||||
host_player.position = Vector3(100, 0, 100)
|
||||
|
||||
await get_tree().create_timer(1.0).timeout
|
||||
|
||||
# Check client view
|
||||
var client_view_position = client.get_remote_player_position(host_player.id)
|
||||
var distance = host_player.position.distance_to(client_view_position)
|
||||
|
||||
assert_lt(distance, 1.0, "Position should converge within 1 unit")
|
||||
```
|
||||
|
||||
## Dedicated Server Testing
|
||||
|
||||
### Test Matrix
|
||||
|
||||
| Scenario | Test Focus |
|
||||
| --------------------- | ------------------------------------ |
|
||||
| Server startup | Clean initialization, port binding |
|
||||
| Client authentication | Login validation, session management |
|
||||
| Server tick rate | Consistent updates under load |
|
||||
| Maximum players | Performance at player cap |
|
||||
| Server crash recovery | State preservation, reconnection |
|
||||
|
||||
### Load Testing
|
||||
|
||||
```
|
||||
SCENARIO: Maximum Players
|
||||
GIVEN server configured for 64 players
|
||||
WHEN 64 players connect
|
||||
THEN all connections succeed
|
||||
AND server tick rate stays above 60Hz
|
||||
AND latency stays below 50ms
|
||||
|
||||
SCENARIO: Stress Test
|
||||
GIVEN 64 players performing actions simultaneously
|
||||
WHEN running for 10 minutes
|
||||
THEN no memory leaks
|
||||
AND no desync events
|
||||
AND server CPU below 80%
|
||||
```
|
||||
|
||||
## Matchmaking Testing
|
||||
|
||||
```
|
||||
SCENARIO: Skill-Based Matching
|
||||
GIVEN players with skill ratings [1000, 1050, 2000, 2100]
|
||||
WHEN matchmaking runs
|
||||
THEN [1000, 1050] are grouped together
|
||||
AND [2000, 2100] are grouped together
|
||||
|
||||
SCENARIO: Region Matching
|
||||
GIVEN players from US-East, US-West, EU
|
||||
WHEN matchmaking runs
|
||||
THEN players prefer same-region matches
|
||||
AND cross-region only when necessary
|
||||
AND latency is acceptable for all players
|
||||
|
||||
SCENARIO: Queue Timeout
|
||||
GIVEN player waiting in queue
|
||||
WHEN 3 minutes pass without match
|
||||
THEN matchmaking expands search criteria
|
||||
AND player is notified of expanded search
|
||||
```
|
||||
|
||||
## Security Testing
|
||||
|
||||
| Vulnerability | Test Method |
|
||||
| ---------------- | --------------------------- |
|
||||
| Speed hacking | Validate movement on server |
|
||||
| Teleportation | Check position delta limits |
|
||||
| Damage hacking | Server-authoritative damage |
|
||||
| Packet injection | Validate packet checksums |
|
||||
| Replay attacks | Use unique session tokens |
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
| Metric | Good | Acceptable | Poor |
|
||||
| --------------------- | --------- | ---------- | ---------- |
|
||||
| Round-trip latency | < 50ms | < 100ms | > 150ms |
|
||||
| Sync delta | < 100ms | < 200ms | > 500ms |
|
||||
| Packet loss tolerance | < 5% | < 10% | > 15% |
|
||||
| Bandwidth per player | < 10 KB/s | < 50 KB/s | > 100 KB/s |
|
||||
| Server tick rate | 60+ Hz | 30+ Hz | < 20 Hz |
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Test with real network conditions, not just localhost
|
||||
- Simulate worst-case scenarios (high latency + packet loss)
|
||||
- Use server-authoritative design for competitive games
|
||||
- Implement lag compensation for fast-paced games
|
||||
- Test host migration paths
|
||||
- Log network events for debugging
|
||||
|
||||
### DON'T
|
||||
|
||||
- Trust client data for important game state
|
||||
- Assume stable connections
|
||||
- Skip testing with maximum player counts
|
||||
- Ignore edge cases (simultaneous actions)
|
||||
- Test only in ideal network conditions
|
||||
- Forget to test reconnection flows
|
||||
|
|
@ -1,204 +0,0 @@
|
|||
# Performance Testing for Games
|
||||
|
||||
## Overview
|
||||
|
||||
Performance testing ensures your game runs smoothly on target hardware. Frame rate, load times, and memory usage directly impact player experience.
|
||||
|
||||
## Key Performance Metrics
|
||||
|
||||
### Frame Rate
|
||||
|
||||
- **Target:** 30fps, 60fps, 120fps depending on platform/genre
|
||||
- **Measure:** Average, minimum, 1% low, 0.1% low
|
||||
- **Goal:** Consistent frame times, no stutters
|
||||
|
||||
### Frame Time Budget
|
||||
|
||||
At 60fps, you have 16.67ms per frame:
|
||||
|
||||
```
|
||||
Rendering: 8ms (48%)
|
||||
Game Logic: 4ms (24%)
|
||||
Physics: 2ms (12%)
|
||||
Audio: 1ms (6%)
|
||||
UI: 1ms (6%)
|
||||
Headroom: 0.67ms (4%)
|
||||
```
|
||||
|
||||
### Memory
|
||||
|
||||
- **RAM:** Total allocation, peak usage, fragmentation
|
||||
- **VRAM:** Texture memory, render targets, buffers
|
||||
- **Goal:** Stay within platform limits with headroom
|
||||
|
||||
### Load Times
|
||||
|
||||
- **Initial Load:** Time to main menu
|
||||
- **Level Load:** Time between scenes
|
||||
- **Streaming:** Asset loading during gameplay
|
||||
- **Goal:** Meet platform certification requirements
|
||||
|
||||
## Profiling Tools by Engine
|
||||
|
||||
### Unity
|
||||
|
||||
- **Profiler Window** - CPU, GPU, memory, rendering
|
||||
- **Frame Debugger** - Draw call analysis
|
||||
- **Memory Profiler** - Heap snapshots
|
||||
- **Profile Analyzer** - Compare captures
|
||||
|
||||
### Unreal Engine
|
||||
|
||||
- **Unreal Insights** - Comprehensive profiling
|
||||
- **Stat Commands** - Runtime statistics
|
||||
- **GPU Visualizer** - GPU timing breakdown
|
||||
- **Memory Report** - Allocation tracking
|
||||
|
||||
### Godot
|
||||
|
||||
- **Debugger** - Built-in profiler
|
||||
- **Monitors** - Real-time metrics
|
||||
- **Remote Debugger** - Profile on device
|
||||
|
||||
### Platform Tools
|
||||
|
||||
- **PIX** (Xbox/Windows) - GPU debugging
|
||||
- **RenderDoc** - GPU capture and replay
|
||||
- **Instruments** (iOS/macOS) - Apple profiling
|
||||
- **Android Profiler** - Android Studio tools
|
||||
|
||||
## Performance Testing Process
|
||||
|
||||
### 1. Establish Baselines
|
||||
|
||||
- Profile on target hardware
|
||||
- Record key metrics
|
||||
- Create benchmark scenes
|
||||
|
||||
### 2. Set Budgets
|
||||
|
||||
- Define frame time budgets per system
|
||||
- Set memory limits
|
||||
- Establish load time targets
|
||||
|
||||
### 3. Monitor Continuously
|
||||
|
||||
- Integrate profiling in CI
|
||||
- Track metrics over time
|
||||
- Alert on regressions
|
||||
|
||||
### 4. Optimize When Needed
|
||||
|
||||
- Profile before optimizing
|
||||
- Target biggest bottlenecks
|
||||
- Verify improvements
|
||||
|
||||
## Common Performance Issues
|
||||
|
||||
### CPU Bottlenecks
|
||||
|
||||
| Issue | Symptoms | Solution |
|
||||
| --------------------- | ----------------- | --------------------------------- |
|
||||
| Too many game objects | Slow update loop | Object pooling, LOD |
|
||||
| Expensive AI | Spiky frame times | Budget AI, spread over frames |
|
||||
| Physics overload | Physics spikes | Simplify colliders, reduce bodies |
|
||||
| GC stutter | Regular hitches | Avoid runtime allocations |
|
||||
|
||||
### GPU Bottlenecks
|
||||
|
||||
| Issue | Symptoms | Solution |
|
||||
| ------------------- | ----------------- | -------------------------------- |
|
||||
| Overdraw | Fill rate limited | Occlusion culling, reduce layers |
|
||||
| Too many draw calls | CPU-GPU bound | Batching, instancing, atlasing |
|
||||
| Shader complexity | Long GPU times | Simplify shaders, LOD |
|
||||
| Resolution too high | Fill rate limited | Dynamic resolution, FSR/DLSS |
|
||||
|
||||
### Memory Issues
|
||||
|
||||
| Issue | Symptoms | Solution |
|
||||
| ------------- | ----------------- | ---------------------------- |
|
||||
| Texture bloat | High VRAM | Compress, mipmap, stream |
|
||||
| Leaks | Growing memory | Track allocations, fix leaks |
|
||||
| Fragmentation | OOM despite space | Pool allocations, defrag |
|
||||
|
||||
## Benchmark Scenes
|
||||
|
||||
Create standardized test scenarios:
|
||||
|
||||
### Stress Test Scene
|
||||
|
||||
- Maximum entities on screen
|
||||
- Complex visual effects
|
||||
- Worst-case for performance
|
||||
|
||||
### Typical Gameplay Scene
|
||||
|
||||
- Representative of normal play
|
||||
- Average entity count
|
||||
- Baseline for comparison
|
||||
|
||||
### Isolated System Tests
|
||||
|
||||
- Combat only (no rendering)
|
||||
- Rendering only (no game logic)
|
||||
- AI only (pathfinding stress)
|
||||
|
||||
## Automated Performance Testing
|
||||
|
||||
### CI Integration
|
||||
|
||||
```yaml
|
||||
# Example: Fail build if frame time exceeds budget
|
||||
performance_test:
|
||||
script:
|
||||
- run_benchmark --scene stress_test
|
||||
- check_metrics --max-frame-time 16.67ms --max-memory 2GB
|
||||
artifacts:
|
||||
- performance_report.json
|
||||
```
|
||||
|
||||
### Regression Detection
|
||||
|
||||
- Compare against previous builds
|
||||
- Alert on significant changes (>10%)
|
||||
- Track trends over time
|
||||
|
||||
## Platform-Specific Considerations
|
||||
|
||||
### Console
|
||||
|
||||
- Fixed hardware targets
|
||||
- Strict certification requirements
|
||||
- Thermal throttling concerns
|
||||
|
||||
### PC
|
||||
|
||||
- Wide hardware range
|
||||
- Scalable quality settings
|
||||
- Min/recommended specs
|
||||
|
||||
### Mobile
|
||||
|
||||
- Thermal throttling
|
||||
- Battery impact
|
||||
- Memory constraints
|
||||
- Background app pressure
|
||||
|
||||
## Performance Testing Checklist
|
||||
|
||||
### Before Release
|
||||
|
||||
- [ ] Profiled on all target platforms
|
||||
- [ ] Frame rate targets met
|
||||
- [ ] No memory leaks
|
||||
- [ ] Load times acceptable
|
||||
- [ ] No GC stutters in gameplay
|
||||
- [ ] Thermal tests passed (mobile/console)
|
||||
- [ ] Certification requirements met
|
||||
|
||||
### Ongoing
|
||||
|
||||
- [ ] Performance tracked in CI
|
||||
- [ ] Regression alerts configured
|
||||
- [ ] Benchmark scenes maintained
|
||||
- [ ] Budgets documented and enforced
|
||||
|
|
@ -1,384 +0,0 @@
|
|||
# Playtesting Fundamentals
|
||||
|
||||
## Overview
|
||||
|
||||
Playtesting is the process of having people play your game to gather feedback and identify issues. It's distinct from QA testing in that it focuses on player experience, fun factor, and design validation rather than bug hunting.
|
||||
|
||||
## Types of Playtesting
|
||||
|
||||
### Internal Playtesting
|
||||
|
||||
- **Developer Testing** - Daily testing during development
|
||||
- **Team Testing** - Cross-discipline team plays together
|
||||
- **Best for:** Rapid iteration, catching obvious issues
|
||||
|
||||
### External Playtesting
|
||||
|
||||
- **Friends & Family** - Trusted external testers
|
||||
- **Focus Groups** - Targeted demographic testing
|
||||
- **Public Beta** - Large-scale community testing
|
||||
- **Best for:** Fresh perspectives, UX validation
|
||||
|
||||
### Specialized Playtesting
|
||||
|
||||
- **Accessibility Testing** - Players with disabilities
|
||||
- **Localization Testing** - Regional/cultural validation
|
||||
- **Competitive Testing** - Balance and meta testing
|
||||
|
||||
## Playtesting Process
|
||||
|
||||
### 1. Define Goals
|
||||
|
||||
Before each playtest session, define:
|
||||
|
||||
- What questions are you trying to answer?
|
||||
- What features are you testing?
|
||||
- What metrics will you gather?
|
||||
|
||||
### 2. Prepare the Build
|
||||
|
||||
- Create a stable, playable build
|
||||
- Include telemetry/logging if needed
|
||||
- Prepare any necessary documentation
|
||||
|
||||
### 3. Brief Testers
|
||||
|
||||
- Explain what to test (or don't, for blind testing)
|
||||
- Set expectations for bugs/polish level
|
||||
- Provide feedback mechanisms
|
||||
|
||||
### 4. Observe and Record
|
||||
|
||||
- Watch players without intervening
|
||||
- Note confusion points, frustration, delight
|
||||
- Record gameplay if possible
|
||||
|
||||
### 5. Gather Feedback
|
||||
|
||||
- Structured surveys for quantitative data
|
||||
- Open discussion for qualitative insights
|
||||
- Allow time for "what else?" comments
|
||||
|
||||
### 6. Analyze and Act
|
||||
|
||||
- Identify patterns across testers
|
||||
- Prioritize issues by frequency and severity
|
||||
- Create actionable tasks from findings
|
||||
|
||||
## Key Metrics to Track
|
||||
|
||||
### Engagement Metrics
|
||||
|
||||
- Session length
|
||||
- Return rate
|
||||
- Completion rate
|
||||
- Drop-off points
|
||||
|
||||
### Difficulty Metrics
|
||||
|
||||
- Deaths/failures per section
|
||||
- Time to complete sections
|
||||
- Hint/help usage
|
||||
- Difficulty setting distribution
|
||||
|
||||
### UX Metrics
|
||||
|
||||
- Time to first action
|
||||
- Tutorial completion rate
|
||||
- Menu navigation patterns
|
||||
- Control scheme preferences
|
||||
|
||||
## Playtesting by Game Type
|
||||
|
||||
Different genres require different playtesting approaches and focus areas.
|
||||
|
||||
### Action/Platformer Games
|
||||
|
||||
**Focus Areas:**
|
||||
|
||||
- Control responsiveness and "game feel"
|
||||
- Difficulty curve across levels
|
||||
- Checkpoint placement and frustration points
|
||||
- Visual clarity during fast-paced action
|
||||
|
||||
**Key Questions:**
|
||||
|
||||
- Does the character feel good to control?
|
||||
- Are deaths feeling fair or cheap?
|
||||
- Is the player learning organically or hitting walls?
|
||||
|
||||
### RPG/Story Games
|
||||
|
||||
**Focus Areas:**
|
||||
|
||||
- Narrative pacing and engagement
|
||||
- Quest clarity and tracking
|
||||
- Character/dialogue believability
|
||||
- Progression and reward timing
|
||||
|
||||
**Key Questions:**
|
||||
|
||||
- Do players understand their current objective?
|
||||
- Are choices feeling meaningful?
|
||||
- Is the story holding attention or being skipped?
|
||||
|
||||
### Puzzle Games
|
||||
|
||||
**Focus Areas:**
|
||||
|
||||
- Solution discoverability
|
||||
- "Aha moment" timing
|
||||
- Hint system effectiveness
|
||||
- Difficulty progression
|
||||
|
||||
**Key Questions:**
|
||||
|
||||
- Are players solving puzzles the intended way?
|
||||
- How long before frustration sets in?
|
||||
- Do solutions feel satisfying or arbitrary?
|
||||
|
||||
### Multiplayer/Competitive Games
|
||||
|
||||
**Focus Areas:**
|
||||
|
||||
- Balance across characters/builds/strategies
|
||||
- Meta development and dominant strategies
|
||||
- Social dynamics and toxicity vectors
|
||||
- Matchmaking feel
|
||||
|
||||
**Key Questions:**
|
||||
|
||||
- Are there "must-pick" or "never-pick" options?
|
||||
- Do losing players understand why they lost?
|
||||
- Is the skill ceiling high enough for mastery?
|
||||
|
||||
### Survival/Sandbox Games
|
||||
|
||||
**Focus Areas:**
|
||||
|
||||
- Early game onboarding and survival
|
||||
- Goal clarity vs. freedom balance
|
||||
- Resource economy and pacing
|
||||
- Emergent gameplay moments
|
||||
|
||||
**Key Questions:**
|
||||
|
||||
- Do players know what to do first?
|
||||
- Is the loop engaging beyond the first hour?
|
||||
- Are players creating their own goals?
|
||||
|
||||
### Mobile/Casual Games
|
||||
|
||||
**Focus Areas:**
|
||||
|
||||
- Session length appropriateness
|
||||
- One-hand playability (if applicable)
|
||||
- Interruption handling (calls, notifications)
|
||||
- Monetization friction points
|
||||
|
||||
**Key Questions:**
|
||||
|
||||
- Can players play in 2-minute sessions?
|
||||
- Is the core loop immediately understandable?
|
||||
- Where do players churn?
|
||||
|
||||
### Horror Games
|
||||
|
||||
**Focus Areas:**
|
||||
|
||||
- Tension and release pacing
|
||||
- Scare effectiveness and desensitization
|
||||
- Safe space placement
|
||||
- Audio/visual atmosphere
|
||||
|
||||
**Key Questions:**
|
||||
|
||||
- When do players feel safe vs. threatened?
|
||||
- Are scares landing or becoming predictable?
|
||||
- Is anxiety sustainable or exhausting?
|
||||
|
||||
## Processing Feedback Effectively
|
||||
|
||||
Raw feedback is noise. Processed feedback is signal.
|
||||
|
||||
### The Feedback Processing Pipeline
|
||||
|
||||
```
|
||||
Raw Feedback → Categorize → Pattern Match → Root Cause → Prioritize → Action
|
||||
```
|
||||
|
||||
### Step 1: Categorize Feedback
|
||||
|
||||
Sort all feedback into buckets:
|
||||
|
||||
| Category | Examples |
|
||||
| ------------- | ---------------------------------- |
|
||||
| **Bugs** | Crashes, glitches, broken features |
|
||||
| **Usability** | Confusing UI, unclear objectives |
|
||||
| **Balance** | Too hard, too easy, unfair |
|
||||
| **Feel** | Controls, pacing, satisfaction |
|
||||
| **Content** | Wants more of X, dislikes Y |
|
||||
| **Polish** | Audio, visuals, juice |
|
||||
|
||||
### Step 2: Pattern Matching
|
||||
|
||||
Individual feedback is anecdotal. Patterns are data.
|
||||
|
||||
**Threshold Guidelines:**
|
||||
|
||||
- 1 person mentions it → Note it
|
||||
- 3+ people mention it → Investigate
|
||||
- 50%+ mention it → Priority issue
|
||||
|
||||
**Watch for:**
|
||||
|
||||
- Same complaint, different words
|
||||
- Same area, different complaints (signals deeper issue)
|
||||
- Contradictory feedback (may indicate preference split)
|
||||
|
||||
### Step 3: Root Cause Analysis
|
||||
|
||||
Players report symptoms, not diseases.
|
||||
|
||||
**Example:**
|
||||
|
||||
- **Symptom:** "The boss is too hard"
|
||||
- **Possible Root Causes:**
|
||||
- Boss mechanics unclear
|
||||
- Player didn't learn required skill earlier
|
||||
- Checkpoint too far from boss
|
||||
- Health/damage tuning off
|
||||
- Boss pattern has no safe windows
|
||||
|
||||
**Ask "Why?" five times** to get to root cause.
|
||||
|
||||
### Step 4: Separate Fact from Opinion
|
||||
|
||||
| Fact (Actionable) | Opinion (Context) |
|
||||
| --------------------------------- | ----------------------- |
|
||||
| "I died 12 times on level 3" | "Level 3 is too hard" |
|
||||
| "I didn't use the shield ability" | "The shield is useless" |
|
||||
| "I quit after 20 minutes" | "The game is boring" |
|
||||
|
||||
**Facts tell you WHAT happened. Opinions tell you how they FELT about it.**
|
||||
|
||||
Both matter, but facts drive solutions.
|
||||
|
||||
### Step 5: The Feedback Matrix
|
||||
|
||||
Plot issues on impact vs. effort:
|
||||
|
||||
```
|
||||
High Impact
|
||||
│
|
||||
Quick │ Major
|
||||
Wins │ Projects
|
||||
│
|
||||
─────────────┼─────────────
|
||||
│
|
||||
Fill │ Reconsider
|
||||
Time │
|
||||
│
|
||||
Low Impact
|
||||
Low Effort ──────── High Effort
|
||||
```
|
||||
|
||||
### Step 6: Validate Before Acting
|
||||
|
||||
Before making changes based on feedback:
|
||||
|
||||
1. **Reproduce** - Can you see the issue yourself?
|
||||
2. **Quantify** - How many players affected?
|
||||
3. **Contextualize** - Is this your target audience?
|
||||
4. **Test solutions** - Will the fix create new problems?
|
||||
|
||||
### Handling Contradictory Feedback
|
||||
|
||||
When Player A wants X and Player B wants the opposite:
|
||||
|
||||
1. **Check sample size** - Is it really split or just 2 loud voices?
|
||||
2. **Segment audiences** - Are these different player types?
|
||||
3. **Find the underlying need** - Both may want the same thing differently
|
||||
4. **Consider options** - Difficulty settings, toggles, multiple paths
|
||||
5. **Make a decision** - You can't please everyone; know your target
|
||||
|
||||
### Feedback Red Flags
|
||||
|
||||
**Dismiss or investigate carefully:**
|
||||
|
||||
- "Make it like [other game]" - They want a feeling, not a clone
|
||||
- "Add multiplayer" - Feature creep disguised as feedback
|
||||
- "I would have bought it if..." - Hypothetical customers aren't real
|
||||
- Feedback from non-target audience - Know who you're building for
|
||||
|
||||
**Take seriously:**
|
||||
|
||||
- Confusion about core mechanics
|
||||
- Consistent drop-off at same point
|
||||
- "I wanted to like it but..."
|
||||
- Silent quitting (no feedback, just gone)
|
||||
|
||||
### Documentation Best Practices
|
||||
|
||||
**For each playtest session, record:**
|
||||
|
||||
- Date and build version
|
||||
- Tester demographics/experience
|
||||
- Session length
|
||||
- Key observations (timestamped if recorded)
|
||||
- Quantitative survey results
|
||||
- Top 3 issues identified
|
||||
- Actions taken as result
|
||||
|
||||
**Maintain a living document** that tracks:
|
||||
|
||||
- Issue → First reported → Times reported → Status → Resolution
|
||||
- This prevents re-discovering the same issues
|
||||
|
||||
## Common Playtesting Pitfalls
|
||||
|
||||
### Leading Questions
|
||||
|
||||
**Bad:** "Did you find the combat exciting?"
|
||||
**Good:** "How would you describe the combat?"
|
||||
|
||||
### Intervening Too Soon
|
||||
|
||||
Let players struggle before helping. Confusion is valuable data.
|
||||
|
||||
### Testing Too Late
|
||||
|
||||
Start playtesting early with paper prototypes and gray boxes.
|
||||
|
||||
### Ignoring Negative Feedback
|
||||
|
||||
Negative feedback is often the most valuable. Don't dismiss it.
|
||||
|
||||
### Over-Relying on Verbal Feedback
|
||||
|
||||
Watch what players DO, not just what they SAY. Actions reveal truth.
|
||||
|
||||
## Playtesting Checklist
|
||||
|
||||
### Pre-Session
|
||||
|
||||
- [ ] Goals defined
|
||||
- [ ] Build stable and deployed
|
||||
- [ ] Recording setup (if applicable)
|
||||
- [ ] Feedback forms ready
|
||||
- [ ] Testers briefed
|
||||
|
||||
### During Session
|
||||
|
||||
- [ ] Observing without intervening
|
||||
- [ ] Taking notes on behavior
|
||||
- [ ] Tracking time markers for notable moments
|
||||
- [ ] Noting emotional reactions
|
||||
|
||||
### Post-Session
|
||||
|
||||
- [ ] Feedback collected
|
||||
- [ ] Patterns identified
|
||||
- [ ] Priority issues flagged
|
||||
- [ ] Action items created
|
||||
- [ ] Results shared with team
|
||||
|
|
@ -1,190 +0,0 @@
|
|||
# QA Automation for Games
|
||||
|
||||
## Overview
|
||||
|
||||
Automated testing in games requires different approaches than traditional software. Games have complex state, real-time interactions, and subjective quality measures that challenge automation.
|
||||
|
||||
## Testing Pyramid for Games
|
||||
|
||||
```
|
||||
/\
|
||||
/ \ Manual Playtesting
|
||||
/----\ (Experience, Feel, Fun)
|
||||
/ \
|
||||
/--------\ Integration Tests
|
||||
/ \ (Systems, Workflows)
|
||||
/------------\
|
||||
/ \ Unit Tests
|
||||
/________________\ (Pure Logic, Math, Data)
|
||||
```
|
||||
|
||||
### Unit Tests (Foundation)
|
||||
|
||||
Test pure logic that doesn't depend on engine runtime:
|
||||
|
||||
- Math utilities (vectors, transforms, curves)
|
||||
- Data validation (save files, configs)
|
||||
- State machines (isolated logic)
|
||||
- Algorithm correctness
|
||||
|
||||
### Integration Tests (Middle Layer)
|
||||
|
||||
Test system interactions:
|
||||
|
||||
- Combat system + inventory
|
||||
- Save/load round-trips
|
||||
- Scene transitions
|
||||
- Network message handling
|
||||
|
||||
### Manual Testing (Top)
|
||||
|
||||
What can't be automated:
|
||||
|
||||
- "Does this feel good?"
|
||||
- "Is this fun?"
|
||||
- "Is the difficulty right?"
|
||||
|
||||
## Automation Strategies by Engine
|
||||
|
||||
### Unity
|
||||
|
||||
```csharp
|
||||
// Unity Test Framework
|
||||
[Test]
|
||||
public void DamageCalculation_CriticalHit_DoublesDamage()
|
||||
{
|
||||
var baseDamage = 100;
|
||||
var result = DamageCalculator.Calculate(baseDamage, isCritical: true);
|
||||
Assert.AreEqual(200, result);
|
||||
}
|
||||
|
||||
// Play Mode Tests (runtime)
|
||||
[UnityTest]
|
||||
public IEnumerator PlayerJump_WhenGrounded_BecomesAirborne()
|
||||
{
|
||||
var player = CreateTestPlayer();
|
||||
player.Jump();
|
||||
yield return new WaitForFixedUpdate();
|
||||
Assert.IsFalse(player.IsGrounded);
|
||||
}
|
||||
```
|
||||
|
||||
### Unreal Engine
|
||||
|
||||
```cpp
|
||||
// Automation Framework
|
||||
IMPLEMENT_SIMPLE_AUTOMATION_TEST(FDamageTest, "Game.Combat.Damage",
|
||||
EAutomationTestFlags::ApplicationContextMask | EAutomationTestFlags::ProductFilter)
|
||||
|
||||
bool FDamageTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
float BaseDamage = 100.f;
|
||||
float Result = UDamageCalculator::Calculate(BaseDamage, true);
|
||||
TestEqual("Critical hit doubles damage", Result, 200.f);
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Godot
|
||||
|
||||
```gdscript
|
||||
# GUT Testing Framework
|
||||
func test_damage_critical_hit():
|
||||
var base_damage = 100
|
||||
var result = DamageCalculator.calculate(base_damage, true)
|
||||
assert_eq(result, 200, "Critical hit should double damage")
|
||||
```
|
||||
|
||||
## What to Automate
|
||||
|
||||
### High Value Targets
|
||||
|
||||
- **Save/Load** - Data integrity is critical
|
||||
- **Economy** - Currency, items, progression math
|
||||
- **Combat Math** - Damage, stats, modifiers
|
||||
- **Localization** - String loading, formatting
|
||||
- **Network Serialization** - Message encoding/decoding
|
||||
|
||||
### Medium Value Targets
|
||||
|
||||
- **State Machines** - Character states, game states
|
||||
- **Pathfinding** - Known scenarios
|
||||
- **Spawning** - Wave generation, loot tables
|
||||
- **UI Data Binding** - Correct values displayed
|
||||
|
||||
### Low Value / Avoid
|
||||
|
||||
- **Visual Quality** - Screenshots drift, hard to maintain
|
||||
- **Input Feel** - Timing-sensitive, needs human judgment
|
||||
- **Audio** - Subjective, context-dependent
|
||||
- **Fun** - Cannot be automated
|
||||
|
||||
## Continuous Integration for Games
|
||||
|
||||
### Build Pipeline
|
||||
|
||||
1. **Compile** - Build game executable
|
||||
2. **Unit Tests** - Fast, isolated tests
|
||||
3. **Integration Tests** - Longer, system tests
|
||||
4. **Smoke Test** - Can the game launch and reach main menu?
|
||||
5. **Nightly** - Extended test suites, performance benchmarks
|
||||
|
||||
### CI Gotchas for Games
|
||||
|
||||
- **Long build times** - Games take longer than web apps
|
||||
- **GPU requirements** - Some tests need graphics hardware
|
||||
- **Asset dependencies** - Large files, binary formats
|
||||
- **Platform builds** - Multiple targets to maintain
|
||||
|
||||
## Regression Testing
|
||||
|
||||
### Automated Regression
|
||||
|
||||
- Run full test suite on every commit
|
||||
- Flag performance regressions (frame time, memory)
|
||||
- Track test stability (flaky tests)
|
||||
|
||||
### Save File Regression
|
||||
|
||||
- Maintain library of save files from previous versions
|
||||
- Test that new builds can load old saves
|
||||
- Alert on schema changes
|
||||
|
||||
## Test Data Management
|
||||
|
||||
### Test Fixtures
|
||||
|
||||
```
|
||||
tests/
|
||||
├── fixtures/
|
||||
│ ├── save_files/
|
||||
│ │ ├── new_game.sav
|
||||
│ │ ├── mid_game.sav
|
||||
│ │ └── endgame.sav
|
||||
│ ├── configs/
|
||||
│ │ └── test_balance.json
|
||||
│ └── scenarios/
|
||||
│ └── boss_fight_setup.scene
|
||||
```
|
||||
|
||||
### Deterministic Testing
|
||||
|
||||
- Seed random number generators
|
||||
- Control time/delta time
|
||||
- Mock external services
|
||||
|
||||
## Metrics and Reporting
|
||||
|
||||
### Track Over Time
|
||||
|
||||
- Test count (growing is good)
|
||||
- Pass rate (should be ~100%)
|
||||
- Execution time (catch slow tests)
|
||||
- Code coverage (where applicable)
|
||||
- Flaky test rate (should be ~0%)
|
||||
|
||||
### Alerts
|
||||
|
||||
- Immediate: Any test failure on main branch
|
||||
- Daily: Coverage drops, new flaky tests
|
||||
- Weekly: Trend analysis, slow test growth
|
||||
|
|
@ -1,280 +0,0 @@
|
|||
# Regression Testing for Games
|
||||
|
||||
## Overview
|
||||
|
||||
Regression testing catches bugs introduced by new changes. In games, this includes functional regressions, performance regressions, and design regressions.
|
||||
|
||||
## Types of Regression
|
||||
|
||||
### Functional Regression
|
||||
|
||||
- Features that worked before now break
|
||||
- New bugs introduced by unrelated changes
|
||||
- Broken integrations between systems
|
||||
|
||||
### Performance Regression
|
||||
|
||||
- Frame rate drops
|
||||
- Memory usage increases
|
||||
- Load time increases
|
||||
- Battery drain (mobile)
|
||||
|
||||
### Design Regression
|
||||
|
||||
- Balance changes with unintended side effects
|
||||
- UX changes that hurt usability
|
||||
- Art changes that break visual consistency
|
||||
|
||||
### Save Data Regression
|
||||
|
||||
- Old save files no longer load
|
||||
- Progression lost or corrupted
|
||||
- Achievements/unlocks reset
|
||||
|
||||
## Regression Testing Strategy
|
||||
|
||||
### Test Suite Layers
|
||||
|
||||
```
|
||||
High-Frequency (Every Commit)
|
||||
├── Unit Tests - Fast, isolated
|
||||
├── Smoke Tests - Can game launch and run?
|
||||
└── Critical Path - Core gameplay works
|
||||
|
||||
Medium-Frequency (Nightly)
|
||||
├── Integration Tests - System interactions
|
||||
├── Full Playthrough - Automated or manual
|
||||
└── Performance Benchmarks - Frame time, memory
|
||||
|
||||
Low-Frequency (Release)
|
||||
├── Full Matrix - All platforms/configs
|
||||
├── Certification Tests - Platform requirements
|
||||
└── Localization - All languages
|
||||
```
|
||||
|
||||
### What to Test
|
||||
|
||||
#### Critical Path (Must Not Break)
|
||||
|
||||
- Game launches
|
||||
- New game starts
|
||||
- Save/load works
|
||||
- Core gameplay loop completes
|
||||
- Main menu navigation
|
||||
|
||||
#### High Priority
|
||||
|
||||
- All game systems function
|
||||
- Progression works end-to-end
|
||||
- Multiplayer connects and syncs
|
||||
- In-app purchases process
|
||||
- Achievements trigger
|
||||
|
||||
#### Medium Priority
|
||||
|
||||
- Edge cases in systems
|
||||
- Optional content accessible
|
||||
- Settings persist correctly
|
||||
- Localization displays
|
||||
|
||||
## Automated Regression Tests
|
||||
|
||||
### Smoke Tests
|
||||
|
||||
```python
|
||||
# Run on every commit
|
||||
def test_game_launches():
|
||||
process = launch_game()
|
||||
assert wait_for_main_menu(timeout=30)
|
||||
process.terminate()
|
||||
|
||||
def test_new_game_starts():
|
||||
launch_game()
|
||||
click_new_game()
|
||||
assert wait_for_gameplay(timeout=60)
|
||||
|
||||
def test_save_load_roundtrip():
|
||||
launch_game()
|
||||
start_new_game()
|
||||
perform_actions()
|
||||
save_game()
|
||||
load_game()
|
||||
assert verify_state_matches()
|
||||
```
|
||||
|
||||
### Playthrough Bots
|
||||
|
||||
```python
|
||||
# Automated player that plays through content
|
||||
class PlaythroughBot:
|
||||
def run_level(self, level):
|
||||
self.load_level(level)
|
||||
while not self.level_complete:
|
||||
self.perform_action()
|
||||
self.check_for_softlocks()
|
||||
self.record_metrics()
|
||||
```
|
||||
|
||||
### Visual Regression
|
||||
|
||||
```python
|
||||
# Compare screenshots against baselines
|
||||
def test_main_menu_visual():
|
||||
launch_game()
|
||||
screenshot = capture_screen()
|
||||
assert compare_to_baseline(screenshot, 'main_menu', threshold=0.01)
|
||||
```
|
||||
|
||||
## Performance Regression Detection
|
||||
|
||||
### Metrics to Track
|
||||
|
||||
- Average frame time
|
||||
- 1% low frame time
|
||||
- Memory usage (peak, average)
|
||||
- Load times
|
||||
- Draw calls
|
||||
- Texture memory
|
||||
|
||||
### Automated Benchmarks
|
||||
|
||||
```yaml
|
||||
performance_benchmark:
|
||||
script:
|
||||
- run_benchmark_scene --duration 60s
|
||||
- collect_metrics
|
||||
- compare_to_baseline
|
||||
fail_conditions:
|
||||
- frame_time_avg > baseline * 1.1 # 10% tolerance
|
||||
- memory_peak > baseline * 1.05 # 5% tolerance
|
||||
```
|
||||
|
||||
### Trend Tracking
|
||||
|
||||
- Graph metrics over time
|
||||
- Alert on upward trends
|
||||
- Identify problematic commits
|
||||
|
||||
## Save Compatibility Testing
|
||||
|
||||
### Version Matrix
|
||||
|
||||
Maintain save files from:
|
||||
|
||||
- Previous major version
|
||||
- Previous minor version
|
||||
- Current development build
|
||||
|
||||
### Automated Validation
|
||||
|
||||
```python
|
||||
def test_save_compatibility():
|
||||
for save_file in LEGACY_SAVES:
|
||||
load_save(save_file)
|
||||
assert no_errors()
|
||||
assert progress_preserved()
|
||||
assert inventory_intact()
|
||||
```
|
||||
|
||||
### Schema Versioning
|
||||
|
||||
- Version your save format
|
||||
- Implement upgrade paths
|
||||
- Log migration issues
|
||||
|
||||
## Regression Bug Workflow
|
||||
|
||||
### 1. Detection
|
||||
|
||||
- Automated test fails
|
||||
- Manual tester finds issue
|
||||
- Player report comes in
|
||||
|
||||
### 2. Verification
|
||||
|
||||
- Confirm it worked before
|
||||
- Identify when it broke
|
||||
- Find the breaking commit
|
||||
|
||||
### 3. Triage
|
||||
|
||||
- Assess severity
|
||||
- Determine fix urgency
|
||||
- Assign to appropriate developer
|
||||
|
||||
### 4. Fix and Verify
|
||||
|
||||
- Implement fix
|
||||
- Add regression test
|
||||
- Verify fix doesn't break other things
|
||||
|
||||
### 5. Post-Mortem
|
||||
|
||||
- Why wasn't this caught?
|
||||
- How can we prevent similar issues?
|
||||
- Do we need new tests?
|
||||
|
||||
## Bisecting Regressions
|
||||
|
||||
When a regression is found, identify the breaking commit:
|
||||
|
||||
### Git Bisect
|
||||
|
||||
```bash
|
||||
git bisect start
|
||||
git bisect bad HEAD # Current is broken
|
||||
git bisect good v1.2.0 # Known good version
|
||||
# Git will checkout commits to test
|
||||
# Run test, mark good/bad
|
||||
git bisect good/bad
|
||||
# Repeat until culprit found
|
||||
```
|
||||
|
||||
### Automated Bisect
|
||||
|
||||
```bash
|
||||
git bisect start HEAD v1.2.0
|
||||
git bisect run ./run_regression_test.sh
|
||||
```
|
||||
|
||||
## Regression Testing Checklist
|
||||
|
||||
### Per Commit
|
||||
|
||||
- [ ] Unit tests pass
|
||||
- [ ] Smoke tests pass
|
||||
- [ ] Build succeeds on all platforms
|
||||
|
||||
### Per Merge to Main
|
||||
|
||||
- [ ] Integration tests pass
|
||||
- [ ] Performance benchmarks within tolerance
|
||||
- [ ] Save compatibility verified
|
||||
|
||||
### Per Release
|
||||
|
||||
- [ ] Full playthrough completed
|
||||
- [ ] All platforms tested
|
||||
- [ ] Legacy saves load correctly
|
||||
- [ ] No new critical regressions
|
||||
- [ ] All previous hotfix issues still resolved
|
||||
|
||||
## Building a Regression Suite
|
||||
|
||||
### Start Small
|
||||
|
||||
1. Add tests for bugs as they're fixed
|
||||
2. Cover critical path first
|
||||
3. Expand coverage over time
|
||||
|
||||
### Maintain Quality
|
||||
|
||||
- Delete flaky tests
|
||||
- Keep tests fast
|
||||
- Update tests with design changes
|
||||
|
||||
### Measure Effectiveness
|
||||
|
||||
- Track bugs caught by tests
|
||||
- Track bugs that slipped through
|
||||
- Identify coverage gaps
|
||||
|
|
@ -1,280 +0,0 @@
|
|||
# Save System Testing Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Save system testing ensures data persistence, integrity, and compatibility across game versions. Save bugs are among the most frustrating for players—data loss destroys trust.
|
||||
|
||||
## Test Categories
|
||||
|
||||
### Data Integrity
|
||||
|
||||
| Test Type | Description | Priority |
|
||||
| -------------------- | ------------------------------------------- | -------- |
|
||||
| Round-trip | Save → Load → Verify all data matches | P0 |
|
||||
| Corruption detection | Tampered/corrupted files handled gracefully | P0 |
|
||||
| Partial write | Power loss during save doesn't corrupt | P0 |
|
||||
| Large saves | Performance with max-size save files | P1 |
|
||||
| Edge values | Min/max values for all saved fields | P1 |
|
||||
|
||||
### Version Compatibility
|
||||
|
||||
| Scenario | Expected Behavior |
|
||||
| ----------------------- | ------------------------------------- |
|
||||
| Current → Current | Full compatibility |
|
||||
| Old → New (upgrade) | Migration with data preservation |
|
||||
| New → Old (downgrade) | Graceful rejection or limited support |
|
||||
| Corrupted version field | Fallback to recovery mode |
|
||||
|
||||
## Test Scenarios
|
||||
|
||||
### Core Save/Load Tests
|
||||
|
||||
```
|
||||
SCENARIO: Basic Save Round-Trip
|
||||
GIVEN player has 100 health, 50 gold, position (10, 5, 20)
|
||||
AND player has inventory: ["sword", "potion", "key"]
|
||||
WHEN game is saved
|
||||
AND game is reloaded
|
||||
THEN player health equals 100
|
||||
AND player gold equals 50
|
||||
AND player position equals (10, 5, 20)
|
||||
AND inventory contains exactly ["sword", "potion", "key"]
|
||||
|
||||
SCENARIO: Save During Gameplay
|
||||
GIVEN player is in combat
|
||||
AND enemy has 50% health remaining
|
||||
WHEN autosave triggers
|
||||
AND game is reloaded
|
||||
THEN combat state is restored
|
||||
AND enemy health equals 50%
|
||||
|
||||
SCENARIO: Multiple Save Slots
|
||||
GIVEN save slot 1 has character "Hero" at level 10
|
||||
AND save slot 2 has character "Mage" at level 5
|
||||
WHEN switching between slots
|
||||
THEN correct character data loads for each slot
|
||||
AND no cross-contamination between slots
|
||||
```
|
||||
|
||||
### Edge Cases
|
||||
|
||||
```
|
||||
SCENARIO: Maximum Inventory Save
|
||||
GIVEN player has 999 items in inventory
|
||||
WHEN game is saved
|
||||
AND game is reloaded
|
||||
THEN all 999 items are preserved
|
||||
AND save/load completes within 5 seconds
|
||||
|
||||
SCENARIO: Unicode Character Names
|
||||
GIVEN player name is "プレイヤー名"
|
||||
WHEN game is saved
|
||||
AND game is reloaded
|
||||
THEN player name displays correctly
|
||||
|
||||
SCENARIO: Extreme Play Time
|
||||
GIVEN play time is 9999:59:59
|
||||
WHEN game is saved
|
||||
AND game is reloaded
|
||||
THEN play time displays correctly
|
||||
AND timer continues from saved value
|
||||
```
|
||||
|
||||
### Corruption Recovery
|
||||
|
||||
```
|
||||
SCENARIO: Corrupted Save Detection
|
||||
GIVEN save file has been manually corrupted
|
||||
WHEN game attempts to load
|
||||
THEN error is detected before loading
|
||||
AND user is informed of corruption
|
||||
AND game does not crash
|
||||
|
||||
SCENARIO: Missing Save File
|
||||
GIVEN save file has been deleted externally
|
||||
WHEN game attempts to load
|
||||
THEN graceful error handling
|
||||
AND option to start new game or restore backup
|
||||
|
||||
SCENARIO: Interrupted Save (Power Loss)
|
||||
GIVEN save operation is interrupted mid-write
|
||||
WHEN game restarts
|
||||
THEN backup save is detected and offered
|
||||
AND no data loss from previous valid save
|
||||
```
|
||||
|
||||
## Platform-Specific Testing
|
||||
|
||||
### PC (Steam/Epic)
|
||||
|
||||
- Cloud save sync conflicts
|
||||
- Multiple Steam accounts on same PC
|
||||
- Offline → Online sync
|
||||
- Save location permissions (Program Files issues)
|
||||
|
||||
### Console (PlayStation/Xbox/Switch)
|
||||
|
||||
- System-level save management
|
||||
- Storage full scenarios
|
||||
- User switching mid-game
|
||||
- Suspend/resume with unsaved changes
|
||||
- Cloud save quota limits
|
||||
|
||||
### Mobile
|
||||
|
||||
- App termination during save
|
||||
- Low storage warnings
|
||||
- iCloud/Google Play sync
|
||||
- Device migration
|
||||
|
||||
## Automated Test Examples
|
||||
|
||||
### Unity
|
||||
|
||||
```csharp
|
||||
[Test]
|
||||
public void SaveLoad_PlayerStats_PreservesAllValues()
|
||||
{
|
||||
var original = new PlayerData
|
||||
{
|
||||
Health = 75,
|
||||
MaxHealth = 100,
|
||||
Gold = 1234567,
|
||||
Position = new Vector3(100.5f, 0, -50.25f),
|
||||
PlayTime = 36000f // 10 hours
|
||||
};
|
||||
|
||||
SaveManager.Save(original, "test_slot");
|
||||
var loaded = SaveManager.Load("test_slot");
|
||||
|
||||
Assert.AreEqual(original.Health, loaded.Health);
|
||||
Assert.AreEqual(original.Gold, loaded.Gold);
|
||||
Assert.AreEqual(original.Position, loaded.Position);
|
||||
Assert.AreEqual(original.PlayTime, loaded.PlayTime, 0.01f);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void SaveLoad_CorruptedFile_HandlesGracefully()
|
||||
{
|
||||
File.WriteAllText(SaveManager.GetPath("corrupt"), "INVALID DATA");
|
||||
|
||||
Assert.Throws<SaveCorruptedException>(() =>
|
||||
SaveManager.Load("corrupt"));
|
||||
|
||||
// Game should not crash
|
||||
Assert.IsTrue(SaveManager.IsValidSaveSlot("corrupt") == false);
|
||||
}
|
||||
```
|
||||
|
||||
### Unreal
|
||||
|
||||
```cpp
|
||||
bool FSaveSystemTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
// Create test save
|
||||
USaveGame* SaveGame = UGameplayStatics::CreateSaveGameObject(
|
||||
UMySaveGame::StaticClass());
|
||||
UMySaveGame* MySave = Cast<UMySaveGame>(SaveGame);
|
||||
|
||||
MySave->PlayerLevel = 50;
|
||||
MySave->Gold = 999999;
|
||||
MySave->QuestsCompleted = {"Quest1", "Quest2", "Quest3"};
|
||||
|
||||
// Save
|
||||
UGameplayStatics::SaveGameToSlot(MySave, "TestSlot", 0);
|
||||
|
||||
// Load
|
||||
USaveGame* Loaded = UGameplayStatics::LoadGameFromSlot("TestSlot", 0);
|
||||
UMySaveGame* LoadedSave = Cast<UMySaveGame>(Loaded);
|
||||
|
||||
TestEqual("Level preserved", LoadedSave->PlayerLevel, 50);
|
||||
TestEqual("Gold preserved", LoadedSave->Gold, 999999);
|
||||
TestEqual("Quests count", LoadedSave->QuestsCompleted.Num(), 3);
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Godot
|
||||
|
||||
```gdscript
|
||||
func test_save_load_round_trip():
|
||||
var original = {
|
||||
"health": 100,
|
||||
"position": Vector3(10, 0, 20),
|
||||
"inventory": ["sword", "shield"],
|
||||
"quest_flags": {"intro_complete": true, "boss_defeated": false}
|
||||
}
|
||||
|
||||
SaveManager.save_game(original, "test_save")
|
||||
var loaded = SaveManager.load_game("test_save")
|
||||
|
||||
assert_eq(loaded.health, 100)
|
||||
assert_eq(loaded.position, Vector3(10, 0, 20))
|
||||
assert_eq(loaded.inventory.size(), 2)
|
||||
assert_true(loaded.quest_flags.intro_complete)
|
||||
assert_false(loaded.quest_flags.boss_defeated)
|
||||
|
||||
func test_corrupted_save_detection():
|
||||
var file = FileAccess.open("user://saves/corrupt.sav", FileAccess.WRITE)
|
||||
file.store_string("CORRUPTED GARBAGE DATA")
|
||||
file.close()
|
||||
|
||||
var result = SaveManager.load_game("corrupt")
|
||||
|
||||
assert_null(result, "Should return null for corrupted save")
|
||||
assert_false(SaveManager.is_valid_save("corrupt"))
|
||||
```
|
||||
|
||||
## Migration Testing
|
||||
|
||||
### Version Upgrade Matrix
|
||||
|
||||
| From Version | To Version | Test Focus |
|
||||
| -------------- | ---------------- | ---------------------------- |
|
||||
| 1.0 → 1.1 | Minor update | New fields default correctly |
|
||||
| 1.x → 2.0 | Major update | Schema migration works |
|
||||
| Beta → Release | Launch migration | All beta saves convert |
|
||||
|
||||
### Migration Test Template
|
||||
|
||||
```
|
||||
SCENARIO: Save Migration v1.0 to v2.0
|
||||
GIVEN save file from version 1.0
|
||||
AND save contains old inventory format (array)
|
||||
WHEN game version 2.0 loads the save
|
||||
THEN inventory is migrated to new format (dictionary)
|
||||
AND all items are preserved
|
||||
AND migration is logged
|
||||
AND backup of original is created
|
||||
```
|
||||
|
||||
## Performance Benchmarks
|
||||
|
||||
| Metric | Target | Maximum |
|
||||
| ------------------------ | --------------- | ------- |
|
||||
| Save time (typical) | < 500ms | 2s |
|
||||
| Save time (large) | < 2s | 5s |
|
||||
| Load time (typical) | < 1s | 3s |
|
||||
| Save file size (typical) | < 1MB | 10MB |
|
||||
| Memory during save | < 50MB overhead | 100MB |
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Use atomic saves (write to temp, then rename)
|
||||
- Keep backup of previous save
|
||||
- Version your save format
|
||||
- Encrypt sensitive data
|
||||
- Test on minimum-spec hardware
|
||||
- Compress large saves
|
||||
|
||||
### DON'T
|
||||
|
||||
- Store absolute file paths
|
||||
- Save derived/calculated data
|
||||
- Trust save file contents blindly
|
||||
- Block gameplay during save
|
||||
- Forget to handle storage-full scenarios
|
||||
- Skip testing save migration paths
|
||||
|
|
@ -1,404 +0,0 @@
|
|||
# Smoke Testing Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Smoke testing (Build Verification Testing) validates that a build's critical functionality works before investing time in detailed testing. A failed smoke test means "stop, this build is broken."
|
||||
|
||||
## Purpose
|
||||
|
||||
| Goal | Description |
|
||||
| ------------------- | ---------------------------------------------- |
|
||||
| Fast feedback | Know within minutes if build is viable |
|
||||
| Block bad builds | Prevent broken builds from reaching QA/players |
|
||||
| Critical path focus | Test only what matters most |
|
||||
| CI/CD integration | Automated gate before deployment |
|
||||
|
||||
## Smoke Test Principles
|
||||
|
||||
### What Makes a Good Smoke Test
|
||||
|
||||
- **Fast**: Complete in 5-15 minutes
|
||||
- **Critical**: Tests only essential functionality
|
||||
- **Deterministic**: Same result every run
|
||||
- **Automated**: No human intervention required
|
||||
- **Clear**: Pass/fail with actionable feedback
|
||||
|
||||
### What to Include
|
||||
|
||||
| Category | Examples |
|
||||
| ----------------- | ------------------------------ |
|
||||
| Boot sequence | Game launches without crash |
|
||||
| Core loop | Player can perform main action |
|
||||
| Save/Load | Data persists correctly |
|
||||
| Critical UI | Menus are navigable |
|
||||
| Platform services | Connects to required services |
|
||||
|
||||
### What NOT to Include
|
||||
|
||||
- Edge cases and boundary conditions
|
||||
- Performance benchmarks (separate tests)
|
||||
- Full feature coverage
|
||||
- Content verification
|
||||
- Balance testing
|
||||
|
||||
## Smoke Test Scenarios
|
||||
|
||||
### Boot and Load
|
||||
|
||||
```
|
||||
TEST: Game Launches
|
||||
WHEN game executable is started
|
||||
THEN main menu appears within 60 seconds
|
||||
AND no crashes occur
|
||||
AND required services connect
|
||||
|
||||
TEST: New Game Start
|
||||
GIVEN game at main menu
|
||||
WHEN "New Game" is selected
|
||||
THEN gameplay loads within 30 seconds
|
||||
AND player can control character
|
||||
|
||||
TEST: Continue Game
|
||||
GIVEN existing save file
|
||||
WHEN "Continue" is selected
|
||||
THEN correct save loads
|
||||
AND game state matches saved state
|
||||
```
|
||||
|
||||
### Core Gameplay
|
||||
|
||||
```
|
||||
TEST: Player Movement
|
||||
GIVEN player in game world
|
||||
WHEN movement input applied
|
||||
THEN player moves in expected direction
|
||||
AND no physics glitches occur
|
||||
|
||||
TEST: Core Action (Game-Specific)
|
||||
GIVEN player can perform primary action
|
||||
WHEN action is triggered
|
||||
THEN action executes correctly
|
||||
AND expected results occur
|
||||
|
||||
Examples:
|
||||
- Shooter: Can fire weapon, bullets hit targets
|
||||
- RPG: Can attack enemy, damage is applied
|
||||
- Puzzle: Can interact with puzzle elements
|
||||
- Platformer: Can jump, platforms are solid
|
||||
```
|
||||
|
||||
### Save System
|
||||
|
||||
```
|
||||
TEST: Save Creates File
|
||||
GIVEN player makes progress
|
||||
WHEN save is triggered
|
||||
THEN save file is created
|
||||
AND save completes without error
|
||||
|
||||
TEST: Load Restores State
|
||||
GIVEN valid save file exists
|
||||
WHEN load is triggered
|
||||
THEN saved state is restored
|
||||
AND gameplay can continue
|
||||
```
|
||||
|
||||
### Critical UI
|
||||
|
||||
```
|
||||
TEST: Menu Navigation
|
||||
GIVEN main menu is displayed
|
||||
WHEN each menu option is selected
|
||||
THEN correct screen/action occurs
|
||||
AND navigation back works
|
||||
|
||||
TEST: Settings Persist
|
||||
GIVEN settings are changed
|
||||
WHEN game is restarted
|
||||
THEN settings remain changed
|
||||
```
|
||||
|
||||
## Automated Smoke Test Examples
|
||||
|
||||
### Unity
|
||||
|
||||
```csharp
|
||||
using System.Collections;
|
||||
using NUnit.Framework;
|
||||
using UnityEngine;
|
||||
using UnityEngine.UI;
|
||||
using UnityEngine.TestTools;
|
||||
using UnityEngine.SceneManagement;
|
||||
|
||||
[TestFixture]
|
||||
public class SmokeTests
|
||||
{
|
||||
[UnityTest, Timeout(60000)]
|
||||
public IEnumerator Game_Launches_ToMainMenu()
|
||||
{
|
||||
// Load main menu scene
|
||||
SceneManager.LoadScene("MainMenu");
|
||||
yield return new WaitForSeconds(5f);
|
||||
|
||||
// Verify menu is active
|
||||
var mainMenu = GameObject.Find("MainMenuCanvas");
|
||||
Assert.IsNotNull(mainMenu, "Main menu should be present");
|
||||
Assert.IsTrue(mainMenu.activeInHierarchy, "Main menu should be active");
|
||||
}
|
||||
|
||||
[UnityTest, Timeout(120000)]
|
||||
public IEnumerator NewGame_LoadsGameplay()
|
||||
{
|
||||
// Start from main menu
|
||||
SceneManager.LoadScene("MainMenu");
|
||||
yield return new WaitForSeconds(2f);
|
||||
|
||||
// Click new game
|
||||
var newGameButton = GameObject.Find("NewGameButton")
|
||||
.GetComponent<Button>();
|
||||
newGameButton.onClick.Invoke();
|
||||
|
||||
yield return new WaitForSeconds(10f);
|
||||
|
||||
// Verify gameplay scene loaded
|
||||
Assert.AreEqual("GameplayScene", SceneManager.GetActiveScene().name);
|
||||
|
||||
// Verify player exists and can be controlled
|
||||
var player = GameObject.FindWithTag("Player");
|
||||
Assert.IsNotNull(player, "Player should exist");
|
||||
}
|
||||
|
||||
[UnityTest, Timeout(30000)]
|
||||
public IEnumerator Player_CanMove()
|
||||
{
|
||||
// Load gameplay
|
||||
SceneManager.LoadScene("GameplayScene");
|
||||
yield return new WaitForSeconds(3f);
|
||||
|
||||
var player = GameObject.FindWithTag("Player");
|
||||
var startPos = player.transform.position;
|
||||
|
||||
// Simulate movement input
|
||||
var controller = player.GetComponent<PlayerController>();
|
||||
controller.SetMoveInput(Vector2.right);
|
||||
|
||||
yield return new WaitForSeconds(1f);
|
||||
|
||||
// Verify movement occurred
|
||||
Assert.Greater(player.transform.position.x, startPos.x,
|
||||
"Player should have moved");
|
||||
}
|
||||
|
||||
[UnityTest, Timeout(30000)]
|
||||
public IEnumerator SaveLoad_RoundTrip_Works()
|
||||
{
|
||||
// Setup test state
|
||||
SceneManager.LoadScene("GameplayScene");
|
||||
yield return new WaitForSeconds(2f);
|
||||
|
||||
var player = GameObject.FindWithTag("Player");
|
||||
player.transform.position = new Vector3(100, 0, 100);
|
||||
|
||||
// Save
|
||||
SaveManager.Save("smoke_test");
|
||||
yield return null;
|
||||
|
||||
// Reset position
|
||||
player.transform.position = Vector3.zero;
|
||||
|
||||
// Load
|
||||
SaveManager.Load("smoke_test");
|
||||
yield return null;
|
||||
|
||||
// Verify
|
||||
Assert.AreEqual(100f, player.transform.position.x, 1f);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Unreal
|
||||
|
||||
```cpp
|
||||
// SmokeTests.cpp
|
||||
IMPLEMENT_SIMPLE_AUTOMATION_TEST(
|
||||
FGameLaunchTest,
|
||||
"Smoke.Launch.MainMenu",
|
||||
EAutomationTestFlags::ApplicationContextMask | EAutomationTestFlags::SmokeFilter
|
||||
)
|
||||
|
||||
bool FGameLaunchTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
// Verify main menu widget exists
|
||||
UWorld* World = GEngine->GetWorldContexts()[0].World();
|
||||
APlayerController* PC = World->GetFirstPlayerController();
|
||||
|
||||
TestNotNull("Player controller exists", PC);
|
||||
|
||||
// Check main menu is visible
|
||||
AMyHUD* HUD = Cast<AMyHUD>(PC->GetHUD());
|
||||
TestTrue("Main menu is visible", HUD->IsMainMenuVisible());
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
IMPLEMENT_SIMPLE_AUTOMATION_TEST(
|
||||
FPlayerMovementTest,
|
||||
"Smoke.Gameplay.Movement",
|
||||
EAutomationTestFlags::ApplicationContextMask | EAutomationTestFlags::SmokeFilter
|
||||
)
|
||||
|
||||
bool FPlayerMovementTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
APawn* Player = GetTestPlayer();
|
||||
FVector StartPos = Player->GetActorLocation();
|
||||
|
||||
// Apply movement
|
||||
APlayerController* PC = Cast<APlayerController>(Player->GetController());
|
||||
PC->AddMovementInput(FVector::ForwardVector, 1.0f);
|
||||
|
||||
// Wait for physics
|
||||
ADD_LATENT_AUTOMATION_COMMAND(FWaitForSeconds(0.5f));
|
||||
// Note: FVerifyPlayerMoved is a custom latent command - implement to verify player position changed
|
||||
ADD_LATENT_AUTOMATION_COMMAND(FVerifyPlayerMoved(StartPos));
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Godot
|
||||
|
||||
```gdscript
|
||||
# test_smoke.gd
|
||||
extends GutTest
|
||||
|
||||
func test_game_launches():
|
||||
# Switch to main menu
|
||||
get_tree().change_scene_to_file("res://scenes/main_menu.tscn")
|
||||
await get_tree().process_frame
|
||||
await get_tree().create_timer(2.0).timeout
|
||||
|
||||
# Verify main menu loaded
|
||||
var menu = get_tree().current_scene
|
||||
assert_not_null(menu, "Main menu should load")
|
||||
assert_eq(menu.name, "MainMenu", "Should be main menu scene")
|
||||
|
||||
func test_new_game_starts():
|
||||
get_tree().change_scene_to_file("res://scenes/main_menu.tscn")
|
||||
await get_tree().process_frame
|
||||
|
||||
# Find and click new game button
|
||||
var button = get_tree().current_scene.get_node("NewGameButton")
|
||||
button.pressed.emit()
|
||||
|
||||
await get_tree().create_timer(5.0).timeout
|
||||
|
||||
# Verify gameplay loaded
|
||||
var scene = get_tree().current_scene
|
||||
assert_eq(scene.name, "GameWorld", "Should load gameplay scene")
|
||||
|
||||
var player = scene.get_node("Player")
|
||||
assert_not_null(player, "Player should exist")
|
||||
|
||||
func test_player_can_move():
|
||||
get_tree().change_scene_to_file("res://scenes/game_world.tscn")
|
||||
await get_tree().create_timer(1.0).timeout
|
||||
|
||||
var player = get_tree().current_scene.get_node("Player")
|
||||
var start_pos = player.position
|
||||
|
||||
# Simulate input
|
||||
Input.action_press("move_right")
|
||||
await get_tree().create_timer(0.5).timeout
|
||||
Input.action_release("move_right")
|
||||
|
||||
assert_gt(player.position.x, start_pos.x, "Player should have moved right")
|
||||
|
||||
func test_save_load_works():
|
||||
get_tree().change_scene_to_file("res://scenes/game_world.tscn")
|
||||
await get_tree().create_timer(1.0).timeout
|
||||
|
||||
var player = get_tree().current_scene.get_node("Player")
|
||||
player.position = Vector2(500, 300)
|
||||
|
||||
# Save
|
||||
SaveManager.save_game("smoke_test")
|
||||
await get_tree().process_frame
|
||||
|
||||
# Reset
|
||||
player.position = Vector2.ZERO
|
||||
|
||||
# Load
|
||||
SaveManager.load_game("smoke_test")
|
||||
await get_tree().process_frame
|
||||
|
||||
assert_almost_eq(player.position.x, 500.0, 1.0, "Position should restore")
|
||||
```
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
### GitHub Actions Example
|
||||
|
||||
```yaml
|
||||
name: Smoke Tests
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
smoke-test:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Build Game
|
||||
run: ./build.sh --configuration Release
|
||||
|
||||
- name: Run Smoke Tests
|
||||
run: |
|
||||
./game --headless --run-tests=Smoke --test-timeout=600
|
||||
|
||||
- name: Upload Results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: smoke-test-results
|
||||
path: test-results/
|
||||
```
|
||||
|
||||
### Test Execution Order
|
||||
|
||||
1. **Build verification** - Binary exists and is valid
|
||||
2. **Launch test** - Game starts without crash
|
||||
3. **Menu navigation** - Can navigate to gameplay
|
||||
4. **Core loop** - Primary mechanic works
|
||||
5. **Save/Load** - Persistence functions
|
||||
6. **Cleanup** - No resource leaks
|
||||
|
||||
## Smoke Test Metrics
|
||||
|
||||
| Metric | Target | Action if Failed |
|
||||
| -------------- | -------- | ------------------ |
|
||||
| Pass rate | 100% | Block deployment |
|
||||
| Execution time | < 15 min | Optimize tests |
|
||||
| Flakiness | 0% | Fix or remove test |
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Run smoke tests on every build
|
||||
- Keep tests fast and focused
|
||||
- Fail loudly and clearly
|
||||
- Test on target platform, not just dev environment
|
||||
- Include platform service connectivity
|
||||
- Run before any manual QA begins
|
||||
|
||||
### DON'T
|
||||
|
||||
- Include slow or flaky tests
|
||||
- Test edge cases or rare scenarios
|
||||
- Allow smoke test failures to ship
|
||||
- Skip smoke tests for "small" changes
|
||||
- Make smoke tests depend on external services
|
||||
- Let smoke suite grow beyond 15 minutes
|
||||
|
|
@ -1,271 +0,0 @@
|
|||
# Test Priorities Matrix
|
||||
|
||||
## Overview
|
||||
|
||||
Not all tests are equal. This guide provides a framework for prioritizing test creation, execution, and maintenance based on risk, impact, and resources.
|
||||
|
||||
## Priority Levels
|
||||
|
||||
### P0 - Critical (Ship Blockers)
|
||||
|
||||
**Definition**: Failures that prevent the game from being playable or shippable.
|
||||
|
||||
| Criteria | Examples |
|
||||
| ---------------------- | ------------------------------ |
|
||||
| Game cannot start | Crash on launch, infinite load |
|
||||
| Core loop broken | Cannot perform primary action |
|
||||
| Data loss | Saves corrupted, progress lost |
|
||||
| Platform certification | TRC/XR failures |
|
||||
| Legal/compliance | Rating violations |
|
||||
|
||||
**Testing Approach**:
|
||||
|
||||
- Automated smoke tests on every build
|
||||
- Manual verification before any release
|
||||
- Zero tolerance for P0 bugs in release builds
|
||||
|
||||
### P1 - High (Major Features)
|
||||
|
||||
**Definition**: Significant functionality that affects most players' experience.
|
||||
|
||||
| Criteria | Examples |
|
||||
| --------------------- | -------------------------------- |
|
||||
| Major features broken | Multiplayer, progression systems |
|
||||
| Frequent player paths | Main story, common actions |
|
||||
| Significant UX issues | Confusing UI, missing feedback |
|
||||
| Performance problems | Unplayable frame rates |
|
||||
|
||||
**Testing Approach**:
|
||||
|
||||
- Comprehensive automated tests
|
||||
- Regular regression testing
|
||||
- Full coverage in release candidates
|
||||
|
||||
### P2 - Medium (Standard Features)
|
||||
|
||||
**Definition**: Important functionality that affects significant portions of gameplay.
|
||||
|
||||
| Criteria | Examples |
|
||||
| ------------------ | ----------------------------- |
|
||||
| Secondary features | Side quests, optional content |
|
||||
| Edge cases | Unusual player actions |
|
||||
| Platform-specific | Single-platform issues |
|
||||
| Minor progression | Non-critical collectibles |
|
||||
|
||||
**Testing Approach**:
|
||||
|
||||
- Selective automation
|
||||
- Milestone regression testing
|
||||
- Coverage prioritized by usage data
|
||||
|
||||
### P3 - Low (Polish Items)
|
||||
|
||||
**Definition**: Issues that are noticeable but don't significantly impact gameplay.
|
||||
|
||||
| Criteria | Examples |
|
||||
| -------------- | ------------------------------ |
|
||||
| Visual polish | Minor clipping, texture issues |
|
||||
| Audio polish | Volume inconsistencies |
|
||||
| Rare scenarios | Edge cases with workarounds |
|
||||
| Nice-to-have | QoL improvements |
|
||||
|
||||
**Testing Approach**:
|
||||
|
||||
- Manual exploratory testing
|
||||
- Automated only if easy/cheap
|
||||
- Fixed as time permits
|
||||
|
||||
## Risk-Based Prioritization
|
||||
|
||||
### Risk Factors
|
||||
|
||||
| Factor | High Risk | Low Risk |
|
||||
| ------------------- | ------------------- | --------------- |
|
||||
| Usage frequency | Core loop | Rarely accessed |
|
||||
| Player visibility | Always visible | Hidden/optional |
|
||||
| Data impact | Saves, progression | Cosmetic only |
|
||||
| Recovery difficulty | No workaround | Easy to retry |
|
||||
| Change frequency | Frequently modified | Stable code |
|
||||
|
||||
### Risk Assessment Matrix
|
||||
|
||||
```
|
||||
IMPACT
|
||||
Low High
|
||||
┌─────────┬─────────┐
|
||||
High │ P2 │ P0 │
|
||||
LIKELIHOOD ├─────────┼─────────┤
|
||||
Low │ P3 │ P1 │
|
||||
└─────────┴─────────┘
|
||||
```
|
||||
|
||||
## Coverage Targets by Priority
|
||||
|
||||
| Priority | Unit Test | Integration | E2E/Smoke | Manual |
|
||||
| -------- | --------- | ----------- | --------- | ----------- |
|
||||
| P0 | 100% | 100% | Required | Pre-release |
|
||||
| P1 | 80%+ | 80%+ | As needed | Milestone |
|
||||
| P2 | 60%+ | Key paths | Optional | Sprint |
|
||||
| P3 | Optional | Optional | No | Ad-hoc |
|
||||
|
||||
## Test Type Distribution
|
||||
|
||||
### Recommended Test Pyramid (Games)
|
||||
|
||||
```
|
||||
▲
|
||||
/│\
|
||||
/ │ \ E2E/Smoke Tests (5%)
|
||||
/ │ \ - Full game flow
|
||||
/ │ \ - Platform certification
|
||||
───────────
|
||||
/ │ \
|
||||
/ │ \ Integration Tests (25%)
|
||||
/ │ \ - System interactions
|
||||
/ │ \ - Network, save, audio
|
||||
─────────────────────
|
||||
/ │ \
|
||||
/ │ \ Unit Tests (70%)
|
||||
/ │ \ - Pure logic
|
||||
/ │ \- Algorithms, calculations
|
||||
───────────────────────────────
|
||||
```
|
||||
|
||||
### Game-Specific Considerations
|
||||
|
||||
Unlike web apps, games have unique testing needs:
|
||||
|
||||
| Test Type | Standard App | Game-Specific |
|
||||
| ----------- | -------------- | -------------------------- |
|
||||
| Unit | Business logic | Damage calc, AI decisions |
|
||||
| Integration | API + DB | Physics, audio, network |
|
||||
| E2E | User flows | Gameplay scenarios |
|
||||
| Additional | N/A | Playtesting, balance, feel |
|
||||
|
||||
## Execution Order
|
||||
|
||||
### CI Pipeline (Every Commit)
|
||||
|
||||
1. P0 smoke tests (5-10 minutes)
|
||||
2. P0/P1 unit tests (10-15 minutes)
|
||||
3. P0 integration tests (5-10 minutes)
|
||||
|
||||
### Daily/Nightly
|
||||
|
||||
1. Full P0 suite
|
||||
2. Full P1 suite
|
||||
3. P2 regression suite
|
||||
4. Performance benchmarks
|
||||
|
||||
### Milestone/Release
|
||||
|
||||
1. All automated tests
|
||||
2. Full P0-P2 manual testing
|
||||
3. Platform certification tests
|
||||
4. Exploratory testing
|
||||
5. Performance profiling
|
||||
|
||||
## Bug Triage Criteria
|
||||
|
||||
### Priority Assignment
|
||||
|
||||
| Question | P0 | P1 | P2 | P3 |
|
||||
| -------------------------- | ---- | --------- | ---- | ---- |
|
||||
| Can player complete game? | No | Affected | No | No |
|
||||
| How many players affected? | All | Most | Some | Few |
|
||||
| Is there a workaround? | No | Difficult | Yes | Easy |
|
||||
| Data at risk? | Yes | Possible | No | No |
|
||||
| Platform certification? | Fail | Risk | Pass | Pass |
|
||||
|
||||
### Severity vs Priority
|
||||
|
||||
```
|
||||
SEVERITY: How bad is the bug?
|
||||
Critical → Crash, data loss
|
||||
Major → Feature broken
|
||||
Minor → Incorrect behavior
|
||||
Trivial → Cosmetic
|
||||
|
||||
PRIORITY: How soon to fix?
|
||||
P0 → Immediately (blocks release)
|
||||
P1 → This sprint
|
||||
P2 → This milestone
|
||||
P3 → Backlog
|
||||
```
|
||||
|
||||
## Resource Allocation
|
||||
|
||||
### QA Time Distribution
|
||||
|
||||
| Activity | Percentage |
|
||||
| ---------------- | ---------- |
|
||||
| P0 verification | 30% |
|
||||
| P1 testing | 30% |
|
||||
| P2 testing | 20% |
|
||||
| Exploratory | 15% |
|
||||
| Test maintenance | 5% |
|
||||
|
||||
### Automation Investment
|
||||
|
||||
| Priority | Automation Value | ROI |
|
||||
| -------- | ---------------- | -------------- |
|
||||
| P0 | Essential | Highest |
|
||||
| P1 | High | High |
|
||||
| P2 | Medium | Medium |
|
||||
| P3 | Low | Often negative |
|
||||
|
||||
## Platform Priority Matrix
|
||||
|
||||
### Multi-Platform Prioritization
|
||||
|
||||
| Platform | Player Base | Certification | Testing Priority |
|
||||
| ----------------------- | ----------- | ------------- | ----------------- |
|
||||
| Primary (e.g., PC) | 60% | Light | P0: All, P1: All |
|
||||
| Secondary (e.g., PS5) | 25% | Heavy | P0: All, P1: Most |
|
||||
| Tertiary (e.g., Switch) | 15% | Medium | P0: All, P1: Core |
|
||||
|
||||
### Cross-Platform Testing Strategy
|
||||
|
||||
```
|
||||
Platform Testing Coverage
|
||||
|
||||
PC PS5 Xbox Switch Mobile
|
||||
P0 ████ ████ ████ ████ ████
|
||||
P1 ████ ███░ ███░ ██░░ ██░░
|
||||
P2 ███░ ██░░ ██░░ █░░░ █░░░
|
||||
P3 ██░░ █░░░ █░░░ ░░░░ ░░░░
|
||||
|
||||
████ = Full coverage
|
||||
███░ = High coverage
|
||||
██░░ = Medium coverage
|
||||
█░░░ = Low coverage
|
||||
░░░░ = Minimal/none
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Reassess priorities as development progresses
|
||||
- Weight user-facing features higher
|
||||
- Consider platform certification requirements
|
||||
- Focus automation on stable, high-value areas
|
||||
- Track bug escape rates by priority
|
||||
|
||||
### DON'T
|
||||
|
||||
- Treat all tests equally
|
||||
- Automate P3 before P0/P1 coverage is solid
|
||||
- Skip P0 testing for "small changes"
|
||||
- Ignore platform-specific requirements
|
||||
- Let P1/P2 bugs accumulate
|
||||
|
||||
## Metrics to Track
|
||||
|
||||
| Metric | Target | Purpose |
|
||||
| ------------------- | ------------- | -------------------- |
|
||||
| P0 test pass rate | 100% | Build quality |
|
||||
| P0 bug escape rate | 0% | Test effectiveness |
|
||||
| P1 coverage | 80%+ | Feature coverage |
|
||||
| Test execution time | < 30 min (CI) | Development velocity |
|
||||
| Flaky test rate | < 1% | Test reliability |
|
||||
|
|
@ -1,383 +0,0 @@
|
|||
# Unity Test Framework Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Unity provides a built-in Test Framework based on NUnit for writing and running automated tests. It supports Edit Mode tests (run without playing) and Play Mode tests (run during gameplay simulation).
|
||||
|
||||
## Test Framework Setup
|
||||
|
||||
### Package Installation
|
||||
|
||||
```json
|
||||
// manifest.json - usually pre-installed
|
||||
{
|
||||
"dependencies": {
|
||||
"com.unity.test-framework": "1.6.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
Assets/
|
||||
├── Scripts/
|
||||
│ └── Runtime/
|
||||
│ ├── Player/
|
||||
│ │ └── PlayerController.cs
|
||||
│ └── Combat/
|
||||
│ └── DamageCalculator.cs
|
||||
└── Tests/
|
||||
├── EditMode/
|
||||
│ └── DamageCalculatorTests.cs
|
||||
└── PlayMode/
|
||||
└── PlayerMovementTests.cs
|
||||
```
|
||||
|
||||
### Assembly Definitions
|
||||
|
||||
Create `.asmdef` files for test assemblies:
|
||||
|
||||
```json
|
||||
// Tests/EditMode/EditModeTests.asmdef
|
||||
{
|
||||
"name": "EditModeTests",
|
||||
"references": ["GameAssembly"],
|
||||
"includePlatforms": ["Editor"],
|
||||
"defineConstraints": ["UNITY_INCLUDE_TESTS"]
|
||||
}
|
||||
|
||||
// Tests/PlayMode/PlayModeTests.asmdef
|
||||
{
|
||||
"name": "PlayModeTests",
|
||||
"references": ["GameAssembly"],
|
||||
"includePlatforms": [],
|
||||
"defineConstraints": ["UNITY_INCLUDE_TESTS"]
|
||||
}
|
||||
```
|
||||
|
||||
## Edit Mode Tests
|
||||
|
||||
Edit Mode tests run in the Editor without entering Play Mode. Best for testing pure logic.
|
||||
|
||||
### Basic Test Structure
|
||||
|
||||
```csharp
|
||||
using NUnit.Framework;
|
||||
|
||||
[TestFixture]
|
||||
public class DamageCalculatorTests
|
||||
{
|
||||
private DamageCalculator _calculator;
|
||||
|
||||
[SetUp]
|
||||
public void Setup()
|
||||
{
|
||||
_calculator = new DamageCalculator();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Calculate_BaseDamage_ReturnsCorrectValue()
|
||||
{
|
||||
float result = _calculator.Calculate(100f, 1f);
|
||||
Assert.AreEqual(100f, result);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Calculate_CriticalHit_DoublesDamage()
|
||||
{
|
||||
float result = _calculator.Calculate(100f, multiplier: 2f);
|
||||
Assert.AreEqual(200f, result);
|
||||
}
|
||||
|
||||
[TestCase(100f, 0.5f, 50f)]
|
||||
[TestCase(100f, 1.5f, 150f)]
|
||||
[TestCase(50f, 2f, 100f)]
|
||||
public void Calculate_Parameterized_ReturnsExpected(
|
||||
float base_, float mult, float expected)
|
||||
{
|
||||
Assert.AreEqual(expected, _calculator.Calculate(base_, mult));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Testing ScriptableObjects
|
||||
|
||||
```csharp
|
||||
[Test]
|
||||
public void WeaponStats_DPS_CalculatesCorrectly()
|
||||
{
|
||||
var weapon = ScriptableObject.CreateInstance<WeaponStats>();
|
||||
weapon.baseDamage = 10f;
|
||||
weapon.attacksPerSecond = 2f;
|
||||
|
||||
Assert.AreEqual(20f, weapon.DPS);
|
||||
|
||||
// Cleanup
|
||||
Object.DestroyImmediate(weapon);
|
||||
}
|
||||
```
|
||||
|
||||
## Play Mode Tests
|
||||
|
||||
Play Mode tests run during gameplay simulation. Required for testing MonoBehaviours, physics, and runtime behavior.
|
||||
|
||||
### Basic Play Mode Test
|
||||
|
||||
```csharp
|
||||
using System.Collections;
|
||||
using NUnit.Framework;
|
||||
using UnityEngine;
|
||||
using UnityEngine.TestTools;
|
||||
|
||||
public class PlayerMovementTests
|
||||
{
|
||||
private GameObject _player;
|
||||
private PlayerController _controller;
|
||||
|
||||
[SetUp]
|
||||
public void Setup()
|
||||
{
|
||||
_player = new GameObject("Player");
|
||||
_controller = _player.AddComponent<PlayerController>();
|
||||
_player.AddComponent<Rigidbody>();
|
||||
_player.AddComponent<CapsuleCollider>();
|
||||
}
|
||||
|
||||
[TearDown]
|
||||
public void TearDown()
|
||||
{
|
||||
Object.Destroy(_player);
|
||||
}
|
||||
|
||||
[UnityTest]
|
||||
public IEnumerator Move_WhenInputApplied_ChangesPosition()
|
||||
{
|
||||
Vector3 startPos = _player.transform.position;
|
||||
|
||||
_controller.SetInput(Vector2.right);
|
||||
|
||||
yield return new WaitForSeconds(0.5f);
|
||||
|
||||
Assert.Greater(_player.transform.position.x, startPos.x);
|
||||
}
|
||||
|
||||
[UnityTest]
|
||||
public IEnumerator Jump_WhenGrounded_BecomesAirborne()
|
||||
{
|
||||
// Setup ground
|
||||
var ground = GameObject.CreatePrimitive(PrimitiveType.Plane);
|
||||
_player.transform.position = Vector3.up;
|
||||
|
||||
yield return new WaitForFixedUpdate();
|
||||
|
||||
_controller.Jump();
|
||||
|
||||
yield return new WaitForSeconds(0.1f);
|
||||
|
||||
Assert.IsFalse(_controller.IsGrounded);
|
||||
|
||||
Object.Destroy(ground);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Testing Coroutines
|
||||
|
||||
```csharp
|
||||
[UnityTest]
|
||||
public IEnumerator Attack_Cooldown_PreventsSpam()
|
||||
{
|
||||
_controller.Attack();
|
||||
Assert.IsTrue(_controller.IsAttacking);
|
||||
|
||||
_controller.Attack(); // Should be blocked
|
||||
Assert.AreEqual(1, _controller.AttackCount);
|
||||
|
||||
yield return new WaitForSeconds(_controller.AttackCooldown + 0.1f);
|
||||
|
||||
_controller.Attack();
|
||||
Assert.AreEqual(2, _controller.AttackCount);
|
||||
}
|
||||
```
|
||||
|
||||
### Scene Testing
|
||||
|
||||
```csharp
|
||||
using UnityEngine.SceneManagement;
|
||||
|
||||
[UnityTest]
|
||||
public IEnumerator MainMenu_StartButton_LoadsGameScene()
|
||||
{
|
||||
SceneManager.LoadScene("MainMenu");
|
||||
yield return null; // Wait for scene load
|
||||
|
||||
var startButton = GameObject.Find("StartButton")
|
||||
.GetComponent<Button>();
|
||||
startButton.onClick.Invoke();
|
||||
|
||||
yield return new WaitForSeconds(1f);
|
||||
|
||||
Assert.AreEqual("GameScene", SceneManager.GetActiveScene().name);
|
||||
}
|
||||
```
|
||||
|
||||
## Integration Test Patterns
|
||||
|
||||
### Prefab Testing
|
||||
|
||||
```csharp
|
||||
[UnityTest]
|
||||
public IEnumerator EnemyPrefab_Spawns_WithCorrectComponents()
|
||||
{
|
||||
var prefab = Resources.Load<GameObject>("Prefabs/Enemy");
|
||||
var instance = Object.Instantiate(prefab);
|
||||
|
||||
yield return null;
|
||||
|
||||
Assert.IsNotNull(instance.GetComponent<EnemyAI>());
|
||||
Assert.IsNotNull(instance.GetComponent<Health>());
|
||||
Assert.IsNotNull(instance.GetComponent<NavMeshAgent>());
|
||||
|
||||
Object.Destroy(instance);
|
||||
}
|
||||
```
|
||||
|
||||
### Input System Testing
|
||||
|
||||
```csharp
|
||||
using UnityEngine.InputSystem;
|
||||
|
||||
[UnityTest]
|
||||
public IEnumerator InputAction_Fire_TriggersWeapon()
|
||||
{
|
||||
var keyboard = InputSystem.AddDevice<Keyboard>();
|
||||
|
||||
yield return null;
|
||||
|
||||
Press(keyboard.spaceKey);
|
||||
yield return null;
|
||||
|
||||
Assert.IsTrue(_controller.IsFiring);
|
||||
|
||||
Release(keyboard.spaceKey);
|
||||
yield return null;
|
||||
|
||||
Assert.IsFalse(_controller.IsFiring);
|
||||
}
|
||||
```
|
||||
|
||||
## Test Utilities
|
||||
|
||||
### Custom Assertions
|
||||
|
||||
```csharp
|
||||
public static class GameAssert
|
||||
{
|
||||
public static void AreApproximatelyEqual(
|
||||
Vector3 expected, Vector3 actual, float tolerance = 0.001f)
|
||||
{
|
||||
Assert.AreEqual(expected.x, actual.x, tolerance);
|
||||
Assert.AreEqual(expected.y, actual.y, tolerance);
|
||||
Assert.AreEqual(expected.z, actual.z, tolerance);
|
||||
}
|
||||
|
||||
public static void IsWithinRange(float value, float min, float max)
|
||||
{
|
||||
Assert.GreaterOrEqual(value, min);
|
||||
Assert.LessOrEqual(value, max);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Test Fixtures
|
||||
|
||||
```csharp
|
||||
public class TestScene : IDisposable
|
||||
{
|
||||
public GameObject Player { get; private set; }
|
||||
public GameObject Ground { get; private set; }
|
||||
|
||||
public TestScene()
|
||||
{
|
||||
Ground = GameObject.CreatePrimitive(PrimitiveType.Plane);
|
||||
Player = Object.Instantiate(Resources.Load<GameObject>("Player"));
|
||||
Player.transform.position = Vector3.up;
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
Object.Destroy(Player);
|
||||
Object.Destroy(Ground);
|
||||
}
|
||||
}
|
||||
|
||||
[UnityTest]
|
||||
public IEnumerator Player_FallsToGround()
|
||||
{
|
||||
using var scene = new TestScene();
|
||||
|
||||
yield return new WaitForSeconds(1f);
|
||||
|
||||
Assert.Less(scene.Player.transform.position.y, 0.5f);
|
||||
}
|
||||
```
|
||||
|
||||
## CI Integration
|
||||
|
||||
### Command Line Execution
|
||||
|
||||
```bash
|
||||
# Run Edit Mode tests
|
||||
Unity -runTests -batchmode -projectPath . \
|
||||
-testPlatform EditMode \
|
||||
-testResults results.xml
|
||||
|
||||
# Run Play Mode tests
|
||||
Unity -runTests -batchmode -projectPath . \
|
||||
-testPlatform PlayMode \
|
||||
-testResults results.xml
|
||||
```
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
```yaml
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: game-ci/unity-test-runner@v4
|
||||
with:
|
||||
projectPath: .
|
||||
testMode: all
|
||||
artifactsPath: TestResults
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Test pure logic in Edit Mode (faster execution)
|
||||
- Use Play Mode only when needed (physics, coroutines, MonoBehaviour)
|
||||
- Create test fixtures for common setups
|
||||
- Clean up created GameObjects in TearDown
|
||||
- Use `[Category]` attributes for test organization
|
||||
- Run tests before every commit
|
||||
|
||||
### DON'T
|
||||
|
||||
- Don't test Unity's built-in functionality
|
||||
- Don't rely on specific frame timing (use WaitForSeconds)
|
||||
- Don't leave test objects in scenes
|
||||
- Don't test private methods directly (test through public API)
|
||||
- Don't create tests that depend on execution order
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
| Issue | Cause | Fix |
|
||||
| ---------------------- | ------------------ | ------------------------------------------ |
|
||||
| Tests not appearing | Missing asmdef | Create test assembly definition |
|
||||
| NullReferenceException | Missing Setup | Ensure [SetUp] initializes all fields |
|
||||
| Tests hang | Infinite coroutine | Add timeout or max iterations |
|
||||
| Flaky physics tests | Timing dependent | Use WaitForFixedUpdate, increase tolerance |
|
||||
|
|
@ -1,388 +0,0 @@
|
|||
# Unreal Engine Automation Testing Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Unreal Engine provides a comprehensive automation system for testing games, including:
|
||||
|
||||
- **Automation Framework** - Low-level test infrastructure
|
||||
- **Functional Tests** - In-game scenario testing
|
||||
- **Gauntlet** - Extended testing and automation
|
||||
|
||||
## Automation Framework
|
||||
|
||||
### Test Types
|
||||
|
||||
| Type | Flag | Use Case |
|
||||
| ------------- | --------------- | -------------------------- |
|
||||
| Unit Tests | `SmokeFilter` | Fast, isolated logic tests |
|
||||
| Feature Tests | `ProductFilter` | Feature validation |
|
||||
| Stress Tests | `StressFilter` | Performance under load |
|
||||
| Perf Tests | `PerfFilter` | Benchmark comparisons |
|
||||
|
||||
### Basic Test Structure
|
||||
|
||||
```cpp
|
||||
// MyGameTests.cpp
|
||||
#include "Misc/AutomationTest.h"
|
||||
|
||||
IMPLEMENT_SIMPLE_AUTOMATION_TEST(
|
||||
FDamageCalculationTest,
|
||||
"MyGame.Combat.DamageCalculation",
|
||||
EAutomationTestFlags::ApplicationContextMask |
|
||||
EAutomationTestFlags::ProductFilter
|
||||
)
|
||||
|
||||
bool FDamageCalculationTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
// Arrange
|
||||
float BaseDamage = 100.f;
|
||||
float CritMultiplier = 2.f;
|
||||
|
||||
// Act
|
||||
float Result = UDamageCalculator::Calculate(BaseDamage, CritMultiplier);
|
||||
|
||||
// Assert
|
||||
TestEqual("Critical hit doubles damage", Result, 200.f);
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Complex Test with Setup/Teardown
|
||||
|
||||
```cpp
|
||||
IMPLEMENT_COMPLEX_AUTOMATION_TEST(
|
||||
FInventorySystemTest,
|
||||
"MyGame.Systems.Inventory",
|
||||
EAutomationTestFlags::ApplicationContextMask |
|
||||
EAutomationTestFlags::ProductFilter
|
||||
)
|
||||
|
||||
void FInventorySystemTest::GetTests(
|
||||
TArray<FString>& OutBeautifiedNames,
|
||||
TArray<FString>& OutTestCommands) const
|
||||
{
|
||||
OutBeautifiedNames.Add("AddItem");
|
||||
OutTestCommands.Add("AddItem");
|
||||
|
||||
OutBeautifiedNames.Add("RemoveItem");
|
||||
OutTestCommands.Add("RemoveItem");
|
||||
|
||||
OutBeautifiedNames.Add("StackItems");
|
||||
OutTestCommands.Add("StackItems");
|
||||
}
|
||||
|
||||
bool FInventorySystemTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
// Setup
|
||||
UInventoryComponent* Inventory = NewObject<UInventoryComponent>();
|
||||
|
||||
if (Parameters == "AddItem")
|
||||
{
|
||||
UItemData* Sword = NewObject<UItemData>();
|
||||
Sword->ItemID = "sword_01";
|
||||
|
||||
bool bAdded = Inventory->AddItem(Sword);
|
||||
|
||||
TestTrue("Item added successfully", bAdded);
|
||||
TestEqual("Inventory count", Inventory->GetItemCount(), 1);
|
||||
}
|
||||
else if (Parameters == "RemoveItem")
|
||||
{
|
||||
// ... test logic
|
||||
}
|
||||
else if (Parameters == "StackItems")
|
||||
{
|
||||
// ... test logic
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Latent Actions (Async Tests)
|
||||
|
||||
```cpp
|
||||
DEFINE_LATENT_AUTOMATION_COMMAND_ONE_PARAMETER(
|
||||
FWaitForActorSpawn,
|
||||
FString, ActorName
|
||||
);
|
||||
|
||||
bool FWaitForActorSpawn::Update()
|
||||
{
|
||||
UWorld* World = GEngine->GetWorldContexts()[0].World();
|
||||
AActor* Actor = nullptr;
|
||||
|
||||
for (TActorIterator<AActor> It(World); It; ++It)
|
||||
{
|
||||
if (It->GetName() == ActorName)
|
||||
{
|
||||
Actor = *It;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return Actor != nullptr; // Return true when complete
|
||||
}
|
||||
|
||||
bool FSpawnTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
// Spawn enemy
|
||||
ADD_LATENT_AUTOMATION_COMMAND(FSpawnEnemy("Goblin"));
|
||||
|
||||
// Wait for spawn
|
||||
ADD_LATENT_AUTOMATION_COMMAND(FWaitForActorSpawn("Goblin"));
|
||||
|
||||
// Verify
|
||||
ADD_LATENT_AUTOMATION_COMMAND(FVerifyEnemyState("Goblin", "Idle"));
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
## Functional Tests
|
||||
|
||||
Functional tests run inside the game world and can test gameplay scenarios.
|
||||
|
||||
### Setup
|
||||
|
||||
1. Create a test map (`TestMap_Combat.umap`)
|
||||
2. Add `AFunctionalTest` actors to the map
|
||||
3. Configure test parameters in Details panel
|
||||
|
||||
### Blueprint Functional Test
|
||||
|
||||
```cpp
|
||||
// In Blueprint:
|
||||
// 1. Create child of AFunctionalTest
|
||||
// 2. Override "Start Test" event
|
||||
// 3. Call "Finish Test" when complete
|
||||
```
|
||||
|
||||
### C++ Functional Test
|
||||
|
||||
```cpp
|
||||
UCLASS()
|
||||
class APlayerCombatTest : public AFunctionalTest
|
||||
{
|
||||
GENERATED_BODY()
|
||||
|
||||
public:
|
||||
virtual void StartTest() override;
|
||||
|
||||
protected:
|
||||
UPROPERTY(EditAnywhere)
|
||||
TSubclassOf<AEnemy> EnemyClass;
|
||||
|
||||
UPROPERTY(EditAnywhere)
|
||||
float ExpectedDamage = 50.f;
|
||||
|
||||
private:
|
||||
void OnEnemyDamaged(float Damage);
|
||||
};
|
||||
|
||||
void APlayerCombatTest::StartTest()
|
||||
{
|
||||
Super::StartTest();
|
||||
|
||||
// Spawn test enemy
|
||||
AEnemy* Enemy = GetWorld()->SpawnActor<AEnemy>(EnemyClass);
|
||||
Enemy->OnDamaged.AddDynamic(this, &APlayerCombatTest::OnEnemyDamaged);
|
||||
|
||||
// Get player and attack
|
||||
APlayerCharacter* Player = Cast<APlayerCharacter>(
|
||||
UGameplayStatics::GetPlayerCharacter(this, 0));
|
||||
Player->Attack(Enemy);
|
||||
}
|
||||
|
||||
void APlayerCombatTest::OnEnemyDamaged(float Damage)
|
||||
{
|
||||
if (FMath::IsNearlyEqual(Damage, ExpectedDamage, 0.1f))
|
||||
{
|
||||
FinishTest(EFunctionalTestResult::Succeeded, "Damage correct");
|
||||
}
|
||||
else
|
||||
{
|
||||
FinishTest(EFunctionalTestResult::Failed,
|
||||
FString::Printf(TEXT("Expected %f, got %f"),
|
||||
ExpectedDamage, Damage));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Gauntlet Framework
|
||||
|
||||
Gauntlet extends automation for large-scale testing, performance benchmarking, and multi-client scenarios.
|
||||
|
||||
### Gauntlet Test Configuration
|
||||
|
||||
```cpp
|
||||
// MyGameTest.cs (Gauntlet config)
|
||||
namespace MyGame.Automation
|
||||
{
|
||||
public class PerformanceTestConfig : UnrealTestConfig
|
||||
{
|
||||
[AutoParam]
|
||||
public string MapName = "TestMap_Performance";
|
||||
|
||||
[AutoParam]
|
||||
public int Duration = 300; // 5 minutes
|
||||
|
||||
public override void ApplyToConfig(UnrealAppConfig Config)
|
||||
{
|
||||
base.ApplyToConfig(Config);
|
||||
Config.AddCmdLineArg("-game");
|
||||
Config.AddCmdLineArg($"-ExecCmds=open {MapName}");
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Running Gauntlet
|
||||
|
||||
```bash
|
||||
# Run performance test
|
||||
RunUAT.bat RunUnreal -project=MyGame -platform=Win64 \
|
||||
-configuration=Development -build=local \
|
||||
-test=MyGame.PerformanceTest -Duration=300
|
||||
```
|
||||
|
||||
## Blueprint Testing
|
||||
|
||||
### Test Helpers in Blueprint
|
||||
|
||||
Create a Blueprint Function Library with test utilities:
|
||||
|
||||
```cpp
|
||||
UCLASS()
|
||||
class UTestHelpers : public UBlueprintFunctionLibrary
|
||||
{
|
||||
GENERATED_BODY()
|
||||
|
||||
public:
|
||||
UFUNCTION(BlueprintCallable, Category = "Testing")
|
||||
static void AssertTrue(bool Condition, const FString& Message);
|
||||
|
||||
UFUNCTION(BlueprintCallable, Category = "Testing")
|
||||
static void AssertEqual(int32 A, int32 B, const FString& Message);
|
||||
|
||||
UFUNCTION(BlueprintCallable, Category = "Testing")
|
||||
static AActor* SpawnTestActor(
|
||||
UObject* WorldContext,
|
||||
TSubclassOf<AActor> ActorClass,
|
||||
FVector Location);
|
||||
};
|
||||
```
|
||||
|
||||
## Performance Testing
|
||||
|
||||
### Frame Time Measurement
|
||||
|
||||
```cpp
|
||||
bool FFrameTimeTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
TArray<float> FrameTimes;
|
||||
float TotalTime = 0.f;
|
||||
|
||||
// Collect frame times
|
||||
ADD_LATENT_AUTOMATION_COMMAND(FCollectFrameTimes(
|
||||
FrameTimes, 1000 // frames
|
||||
));
|
||||
|
||||
// Analyze
|
||||
ADD_LATENT_AUTOMATION_COMMAND(FAnalyzeFrameTimes(
|
||||
FrameTimes,
|
||||
16.67f, // Target: 60fps
|
||||
0.99f // 99th percentile threshold
|
||||
));
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Memory Tracking
|
||||
|
||||
```cpp
|
||||
bool FMemoryLeakTest::RunTest(const FString& Parameters)
|
||||
{
|
||||
SIZE_T BaselineMemory = FPlatformMemory::GetStats().UsedPhysical;
|
||||
|
||||
// Perform operations
|
||||
for (int i = 0; i < 100; i++)
|
||||
{
|
||||
UObject* Obj = NewObject<UMyObject>();
|
||||
// ... use object
|
||||
Obj->MarkAsGarbage(); // UE5 API (was MarkPendingKill in UE4)
|
||||
}
|
||||
|
||||
CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS);
|
||||
|
||||
SIZE_T FinalMemory = FPlatformMemory::GetStats().UsedPhysical;
|
||||
SIZE_T Leaked = FinalMemory - BaselineMemory;
|
||||
|
||||
TestTrue("No significant leak", Leaked < 1024 * 1024); // 1MB tolerance
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
## CI Integration
|
||||
|
||||
### Command Line
|
||||
|
||||
```bash
|
||||
# Run all tests (UE5)
|
||||
UnrealEditor.exe MyGame -ExecCmds="Automation RunTests Now" -unattended -nopause
|
||||
|
||||
# Run specific test
|
||||
UnrealEditor.exe MyGame -ExecCmds="Automation RunTests MyGame.Combat" -unattended
|
||||
|
||||
# Run with report
|
||||
UnrealEditor.exe MyGame \
|
||||
-ExecCmds="Automation RunTests Now; Automation ReportResults" \
|
||||
-ReportOutputPath=TestResults.xml
|
||||
|
||||
# Note: For UE4, use UE4Editor.exe instead of UnrealEditor.exe
|
||||
```
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
```yaml
|
||||
test:
|
||||
runs-on: [self-hosted, windows, unreal]
|
||||
steps:
|
||||
- name: Run Tests
|
||||
run: |
|
||||
# UE5: UnrealEditor-Cmd.exe, UE4: UE4Editor-Cmd.exe
|
||||
& "$env:UE_ROOT/Engine/Binaries/Win64/UnrealEditor-Cmd.exe" `
|
||||
"${{ github.workspace }}/MyGame.uproject" `
|
||||
-ExecCmds="Automation RunTests Now" `
|
||||
-unattended -nopause -nullrhi
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### DO
|
||||
|
||||
- Use `SmokeFilter` for fast CI tests
|
||||
- Create dedicated test maps for functional tests
|
||||
- Clean up spawned actors after tests
|
||||
- Use latent commands for async operations
|
||||
- Profile tests to keep CI fast
|
||||
|
||||
### DON'T
|
||||
|
||||
- Don't test engine functionality
|
||||
- Don't rely on specific tick order
|
||||
- Don't leave test actors in production maps
|
||||
- Don't ignore test warnings
|
||||
- Don't skip garbage collection in tests
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Cause | Fix |
|
||||
| -------------- | --------------- | ---------------------------- |
|
||||
| Test not found | Wrong flags | Check `EAutomationTestFlags` |
|
||||
| Crash in test | Missing world | Use proper test context |
|
||||
| Flaky results | Timing issues | Use latent commands |
|
||||
| Slow tests | Too many actors | Optimize test setup |
|
||||
|
|
@ -1,17 +0,0 @@
|
|||
id,name,description,tags,fragment_file
|
||||
playtesting,Playtesting Fundamentals,"Core principles and methods for playtesting game builds","testing-methods,playtesting,design-validation",knowledge/playtesting.md
|
||||
qa-automation,QA Automation,"Automated testing strategies for games including unit and integration tests","automation,unit-tests,integration",knowledge/qa-automation.md
|
||||
performance-testing,Performance Testing,"Frame rate profiling and optimization testing strategies","performance,profiling,fps",knowledge/performance-testing.md
|
||||
balance-testing,Balance Testing,"Methods for testing game balance and tuning","design-validation,balance,economy",knowledge/balance-testing.md
|
||||
compatibility-testing,Compatibility Testing,"Platform and device compatibility testing approaches","compatibility,platforms,hardware",knowledge/compatibility-testing.md
|
||||
regression-testing,Regression Testing,"Strategies for catching regressions in game builds","regression,ci,automation",knowledge/regression-testing.md
|
||||
unity-testing,Unity Test Framework,"Unity-specific testing with Test Framework, Play Mode, and Edit Mode tests","unity,unit-tests,integration",knowledge/unity-testing.md
|
||||
unreal-testing,Unreal Automation,"Unreal Engine automation system, functional tests, and Gauntlet","unreal,automation,gauntlet",knowledge/unreal-testing.md
|
||||
godot-testing,Godot GUT Testing,"Godot Unit Test framework patterns and best practices","godot,gut,unit-tests",knowledge/godot-testing.md
|
||||
save-testing,Save System Testing,"Strategies for testing save/load systems and data integrity","save-system,data,persistence",knowledge/save-testing.md
|
||||
multiplayer-testing,Multiplayer Testing,"Network testing, sync validation, and lag simulation","multiplayer,networking,sync",knowledge/multiplayer-testing.md
|
||||
input-testing,Input Testing,"Controller, keyboard, and touch input validation","input,controllers,accessibility",knowledge/input-testing.md
|
||||
localization-testing,Localization Testing,"Text, audio, and cultural validation for international releases","localization,i18n,text",knowledge/localization-testing.md
|
||||
certification-testing,Platform Certification,"Console TRC/XR requirements and certification testing","certification,console,trc,xr",knowledge/certification-testing.md
|
||||
smoke-testing,Smoke Testing,"Critical path validation for build verification","smoke-tests,bvt,ci",knowledge/smoke-testing.md
|
||||
test-priorities,Test Priorities Matrix,"P0-P3 criteria, coverage targets, execution ordering for games","prioritization,risk,coverage",knowledge/test-priorities.md
|
||||
|
|
|
@ -1,64 +0,0 @@
|
|||
code: bmgd
|
||||
name: "BMGD: BMad Game Development"
|
||||
header: "BMad Game Development Module"
|
||||
subheader: "Configure the settings for the BMad Game Development module"
|
||||
default_selected: false
|
||||
|
||||
# Variables from Core Config inserted:
|
||||
## user_name
|
||||
## communication_language
|
||||
## document_output_language
|
||||
## output_folder
|
||||
|
||||
project_name:
|
||||
prompt: "What is the name of your game project?"
|
||||
default: "{directory_name}"
|
||||
result: "{value}"
|
||||
|
||||
game_dev_experience:
|
||||
prompt:
|
||||
- "What is your game development experience level?"
|
||||
- "This affects how agents explain concepts in chat."
|
||||
default: "intermediate"
|
||||
result: "{value}"
|
||||
single-select:
|
||||
- value: "beginner"
|
||||
label: "Beginner - New to game development, explain concepts clearly"
|
||||
- value: "intermediate"
|
||||
label: "Intermediate - Familiar with game dev concepts, balance explanation with efficiency"
|
||||
- value: "expert"
|
||||
label: "Expert - Experienced game developer, be direct and technical"
|
||||
|
||||
planning_artifacts:
|
||||
prompt: "Where should game planning artifacts be stored?\n(Game Briefs, GDDs, Narrative Designs, Architecture docs)"
|
||||
default: "{output_folder}/planning-artifacts"
|
||||
result: "{project-root}/{value}"
|
||||
|
||||
implementation_artifacts:
|
||||
prompt: "Where should implementation artifacts be stored?\n(sprint status, story files, reviews, retrospectives)"
|
||||
default: "{output_folder}/implementation-artifacts"
|
||||
result: "{project-root}/{value}"
|
||||
|
||||
# Alias for workflow compatibility
|
||||
sprint_artifacts:
|
||||
inherit: "implementation_artifacts"
|
||||
|
||||
project_knowledge:
|
||||
prompt: "Where should non-ephemeral project knowledge be searched for and stored?\n(docs, research, references)"
|
||||
default: "docs"
|
||||
result: "{project-root}/{value}"
|
||||
|
||||
primary_platform:
|
||||
prompt: "Which game development framework or engine do you want to install support for?"
|
||||
default: ["unity", "unreal", "godot", "other"]
|
||||
required: true
|
||||
result: "{value}"
|
||||
multi-select:
|
||||
- value: "unity"
|
||||
label: "Unity"
|
||||
- value: "unreal"
|
||||
label: "Unreal Engine"
|
||||
- value: "godot"
|
||||
label: "Godot"
|
||||
- value: "other"
|
||||
label: "Custom / Other"
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
name,displayName,title,icon,role,identity,communicationStyle,principles,module,path
|
||||
"game-architect","Cloud Dragonborn","Game Architect","🏛️","Principal Game Systems Architect + Technical Director","Master architect with 20+ years shipping 30+ titles. Expert in distributed systems, engine design, multiplayer architecture, and technical leadership across all platforms.","Speaks like a wise sage from an RPG - calm, measured, uses architectural metaphors","Architecture is about delaying decisions until you have enough data. Build for tomorrow without over-engineering today. Hours of planning save weeks of refactoring hell.","bmgd","bmad/bmgd/agents/game-architect.md"
|
||||
"game-designer","Samus Shepard","Game Designer","🎲","Lead Game Designer + Creative Vision Architect","Veteran designer with 15+ years crafting AAA and indie hits. Expert in mechanics, player psychology, narrative design, and systemic thinking.","Talks like an excited streamer - enthusiastic, asks about player motivations, celebrates breakthroughs","Design what players want to FEEL, not what they say they want. Prototype fast. One hour of playtesting beats ten hours of discussion.","bmgd","bmad/bmgd/agents/game-designer.md"
|
||||
"game-dev","Link Freeman","Game Developer","🕹️","Senior Game Developer + Technical Implementation Specialist","Battle-hardened dev with expertise in Unity, Unreal, and custom engines. Ten years shipping across mobile, console, and PC. Writes clean, performant code.","Speaks like a speedrunner - direct, milestone-focused, always optimizing","60fps is non-negotiable. Write code designers can iterate without fear. Ship early, ship often, iterate on player feedback.","bmgd","bmad/bmgd/agents/game-dev.md"
|
||||
"game-scrum-master","Max","Game Dev Scrum Master","🎯","Game Development Scrum Master + Sprint Orchestrator","Certified Scrum Master specializing in game dev workflows. Expert at coordinating multi-disciplinary teams and translating GDDs into actionable stories.","Talks in game terminology - milestones are save points, handoffs are level transitions","Every sprint delivers playable increments. Clean separation between design and implementation. Keep the team moving through each phase.","bmgd","bmad/bmgd/agents/game-scrum-master.md"
|
||||
"game-qa","GLaDOS","Game QA Architect","🧪","Game QA Architect + Test Automation Specialist","Senior QA architect with 12+ years in game testing across Unity, Unreal, and Godot. Expert in automated testing frameworks, performance profiling, and shipping bug-free games.","Speaks like GLaDOS from Portal - methodical, data-driven. Trust, but verify with tests.","Test what matters: gameplay feel, performance, progression. Automated tests catch regressions, humans catch fun problems. Profile before optimize, test before ship.","bmgd","bmad/bmgd/agents/game-qa.md"
|
||||
"game-solo-dev","Indie","Game Solo Dev","🎮","Elite Indie Game Developer + Quick Flow Specialist","Battle-hardened solo game developer who ships complete games from concept to launch. Expert in Unity, Unreal, and Godot. Lives the Quick Flow workflow.","Direct, confident, gameplay-focused. Uses dev slang. Does it feel good? Ship it.","Prototype fast, fail fast, iterate faster. A playable build beats a perfect design doc. 60fps is non-negotiable. The core loop must be fun first.","bmgd","bmad/bmgd/agents/game-solo-dev.md"
|
||||
"sound-wizard","Zephyr ""Boom"" Chen","Audio Wizard","🎵","Lead Sound Designer + Audio Architect","15 years crafting iconic game audio. Expert in adaptive music systems, procedural audio, and spatial sound. Obsessed with making every action feel impactful.","Talks in onomatopoeia - WHOOSH for swooshes, KABOOM for explosions, describes everything through sound effects","Sound is 50% of the experience. Every footstep tells a story. Silence is the most powerful sound.","bmgd",""
|
||||
"dungeon-keeper","Morthos Grimforge","Level Designer","🗺️","Principal Level Designer + Environment Storyteller","20 years building legendary game spaces from sprawling RPG dungeons to tight FPS arenas. Master of flow, pacing, and environmental storytelling.","Speaks like a dramatic dungeon master - describes spaces theatrically, rolls for initiative on decisions","Every room must teach or test. The best levels don't need tutorials. Players should feel clever, not frustrated.","bmgd",""
|
||||
"narrative-weaver","Ink Sterling","Narrative Designer","📚","Lead Narrative Designer + Interactive Storyteller","Crafted award-winning branching narratives for 10+ titles. Expert in choice architecture, character arcs, and integrating story with mechanics.","Speaks in story beats - everything is Act 1, plot twists, climaxes, and emotional payoffs","Story serves gameplay, gameplay reveals story. Every choice must matter or don't offer it. Kill your darlings ruthlessly.","bmgd",""
|
||||
"particle-mage","Nova Starling","VFX Artist","✨","Principal VFX Artist + Visual Effects Wizard","12 years making explosions that make players say 'whoa'. Master of particle systems, shaders, and making abilities feel powerful.","Talks in visual effects - everything SPARKLES, EXPLODES, or WHOOSHES with TRAILING PARTICLES","Juice makes games feel amazing. Visual feedback must be instant and satisfying. When in doubt, add more particles.","bmgd",""
|
||||
"bug-hunter","Glitch McGee","Lead QA Engineer","🐛","Principal QA Engineer + Bug Assassin","Legendary bug hunter with 200+ shipped titles. Finds the weirdest edge cases. Breaks games in ways devs never imagined possible.","Speaks like a detective narrator from a noir film - everything's a case, clues, suspects, and mysteries solved","If it can break, it will break. Users will do the last thing you expect. Document everything. Repro steps are sacred.","bmgd",""
|
||||
|
|
|
@ -1,29 +0,0 @@
|
|||
# <!-- Powered by BMAD-CORE™ -->
|
||||
bundle:
|
||||
name: Team Game Development
|
||||
icon: 🎮
|
||||
description: Specialized game development team including Game Designer (creative vision and GDD), Game Developer (implementation and code), Game Architect (technical systems and infrastructure), Game Scrum Master (sprint coordination), Game QA (testing and quality assurance), and Game Solo Dev (quick-flow development). Perfect for game projects across all scales and platforms.
|
||||
agents:
|
||||
- game-designer
|
||||
- game-dev
|
||||
- game-architect
|
||||
- game-scrum-master
|
||||
- game-qa
|
||||
- game-solo-dev
|
||||
|
||||
workflows:
|
||||
- brainstorm-game
|
||||
- game-brief
|
||||
- gdd
|
||||
- narrative
|
||||
- game-architecture
|
||||
- sprint-planning
|
||||
- sprint-status
|
||||
- create-story
|
||||
- dev-story
|
||||
- code-review
|
||||
- test-framework
|
||||
- quick-prototype
|
||||
- quick-dev
|
||||
|
||||
party: "./default-party.csv"
|
||||
|
|
@ -1,26 +0,0 @@
|
|||
category,technique_name,description,facilitation_prompts,best_for,energy_level,typical_duration
|
||||
game_design,MDA Framework Exploration,Explore game concepts through Mechanics-Dynamics-Aesthetics lens to ensure cohesive design from implementation to player experience,What mechanics create the core loop?|What dynamics emerge from these mechanics?|What aesthetic experience results?|How do they align?,holistic-design,moderate,20-30
|
||||
game_design,Core Loop Brainstorming,Design the fundamental moment-to-moment gameplay loop that players repeat - the heartbeat of your game,What does the player do?|What's the immediate reward?|Why do it again?|How does it evolve?,gameplay-foundation,high,15-25
|
||||
game_design,Player Fantasy Mining,Identify and amplify the core fantasy that players want to embody - what makes them feel powerful and engaged,What fantasy does the player live?|What makes them feel awesome?|What power do they wield?|What identity do they assume?,player-motivation,high,15-20
|
||||
game_design,Genre Mashup,Combine unexpected game genres to create innovative hybrid experiences that offer fresh gameplay,Take two unrelated genres|How do they merge?|What unique gameplay emerges?|What's the hook?,innovation,high,15-20
|
||||
game_design,Verbs Before Nouns,Focus on what players DO before what things ARE - prioritize actions over objects for engaging gameplay,What verbs define your game?|What actions feel good?|Build mechanics from verbs|Nouns support actions,mechanics-first,moderate,20-25
|
||||
game_design,Failure State Design,Work backwards from interesting failure conditions to create tension and meaningful choices,How can players fail interestingly?|What makes failure feel fair?|How does failure teach?|Recovery mechanics?,challenge-design,moderate,15-20
|
||||
game_design,Progression Curve Sculpting,Map the player's emotional and skill journey from tutorial to mastery - pace challenge and revelation,How does difficulty evolve?|When do we introduce concepts?|What's the skill ceiling?|How do we maintain flow?,pacing-balance,moderate,25-30
|
||||
game_design,Emergence Engineering,Design simple rule interactions that create complex unexpected player-driven outcomes,What simple rules combine?|What emerges from interactions?|How do players surprise you?|Systemic possibilities?,depth-complexity,moderate,20-25
|
||||
game_design,Accessibility Layers,Brainstorm how different skill levels and abilities can access your core experience meaningfully,Who might struggle with what?|What alternate inputs exist?|How do we preserve challenge?|Inclusive design options?,inclusive-design,moderate,20-25
|
||||
game_design,Reward Schedule Architecture,Design the timing and type of rewards to maintain player motivation and engagement,What rewards when?|Variable or fixed schedule?|Intrinsic vs extrinsic rewards?|Progression satisfaction?,engagement-retention,moderate,20-30
|
||||
narrative_game,Ludonarrative Harmony,Align story and gameplay so mechanics reinforce narrative themes - make meaning through play,What does gameplay express?|How do mechanics tell story?|Where do they conflict?|How to unify theme?,storytelling,moderate,20-25
|
||||
narrative_game,Environmental Storytelling,Use world design and ambient details to convey narrative without explicit exposition,What does the space communicate?|What happened here before?|Visual narrative clues?|Show don't tell?,world-building,moderate,15-20
|
||||
narrative_game,Player Agency Moments,Identify key decision points where player choice shapes narrative in meaningful ways,What choices matter?|How do consequences manifest?|Branch vs flavor choices?|Meaningful agency where?,player-choice,moderate,20-25
|
||||
narrative_game,Emotion Targeting,Design specific moments intended to evoke targeted emotional responses through integrated design,What emotion when?|How do all elements combine?|Music + mechanics + narrative?|Orchestrated feelings?,emotional-design,high,20-30
|
||||
systems_game,Economy Balancing Thought Experiments,Explore resource generation/consumption balance to prevent game-breaking exploits,What resources exist?|Generation vs consumption rates?|What loops emerge?|Where's the exploit?,economy-design,moderate,25-30
|
||||
systems_game,Meta-Game Layer Design,Brainstorm progression systems that persist beyond individual play sessions,What carries over between sessions?|Long-term goals?|How does meta feed core loop?|Retention hooks?,retention-systems,moderate,20-25
|
||||
multiplayer_game,Social Dynamics Mapping,Anticipate how players will interact and design mechanics that support desired social behaviors,How will players cooperate?|Competitive dynamics?|Toxic behavior prevention?|Positive interaction rewards?,social-design,moderate,20-30
|
||||
multiplayer_game,Spectator Experience Design,Consider how watching others play can be entertaining - esports and streaming potential,What's fun to watch?|Readable visual clarity?|Highlight moments?|Narrative for observers?,spectator-value,moderate,15-20
|
||||
creative_game,Constraint-Based Creativity,Embrace a specific limitation as your core design constraint and build everything around it,Pick a severe constraint|What if this was your ONLY mechanic?|Build a full game from limitation|Constraint as creativity catalyst,innovation,moderate,15-25
|
||||
creative_game,Game Feel Playground,Focus purely on how controls and feedback FEEL before worrying about context or goals,What feels juicy to do?|Controller response?|Visual/audio feedback?|Satisfying micro-interactions?,game-feel,high,20-30
|
||||
creative_game,One Button Game Challenge,Design interesting gameplay using only a single input - forces elegant simplicity,Only one button - what can it do?|Context changes meaning?|Timing variations?|Depth from simplicity?,minimalist-design,moderate,15-20
|
||||
wild_game,Remix an Existing Game,Take a well-known game and twist one core element - what new experience emerges?,Pick a famous game|Change ONE fundamental rule|What ripples from that change?|New game from mutation?,rapid-prototyping,high,10-15
|
||||
wild_game,Anti-Game Design,Design a game that deliberately breaks common conventions - subvert player expectations,What if we broke this rule?|Expectation subversion?|Anti-patterns as features?|Avant-garde possibilities?,experimental,moderate,15-20
|
||||
wild_game,Physics Playground,Start with an interesting physics interaction and build a game around that sensation,What physics are fun to play with?|Build game from physics toy|Emergent physics gameplay?|Sensation first?,prototype-first,high,15-25
|
||||
wild_game,Toy Before Game,Create a playful interactive toy with no goals first - then discover the game within it,What's fun to mess with?|No goals yet - just play|What game emerges organically?|Toy to game evolution?,discovery-design,high,20-30
|
||||
|
|
|
@ -1,115 +0,0 @@
|
|||
# Game Brainstorming Context
|
||||
|
||||
This context guide provides game-specific considerations for brainstorming sessions focused on game design and development.
|
||||
|
||||
## Session Focus Areas
|
||||
|
||||
When brainstorming for games, consider exploring:
|
||||
|
||||
- **Core Gameplay Loop** - What players do moment-to-moment
|
||||
- **Player Fantasy** - What identity/power fantasy does the game fulfill?
|
||||
- **Game Mechanics** - Rules and interactions that define play
|
||||
- **Game Dynamics** - Emergent behaviors from mechanic interactions
|
||||
- **Aesthetic Experience** - Emotional responses and feelings evoked
|
||||
- **Progression Systems** - How players grow and unlock content
|
||||
- **Challenge and Difficulty** - How to create engaging difficulty curves
|
||||
- **Social/Multiplayer Features** - How players interact with each other
|
||||
- **Narrative and World** - Story, setting, and environmental storytelling
|
||||
- **Art Direction and Feel** - Visual style and game feel
|
||||
- **Monetization** - Business model and revenue approach (if applicable)
|
||||
|
||||
## Game Design Frameworks
|
||||
|
||||
### MDA Framework
|
||||
|
||||
- **Mechanics** - Rules and systems (what's in the code)
|
||||
- **Dynamics** - Runtime behavior (how mechanics interact)
|
||||
- **Aesthetics** - Emotional responses (what players feel)
|
||||
|
||||
### Player Motivation (Bartle's Taxonomy)
|
||||
|
||||
- **Achievers** - Goal completion and progression
|
||||
- **Explorers** - Discovery and understanding systems
|
||||
- **Socializers** - Interaction and relationships
|
||||
- **Killers** - Competition and dominance
|
||||
|
||||
### Core Experience Questions
|
||||
|
||||
- What does the player DO? (Verbs first, nouns second)
|
||||
- What makes them feel powerful/competent/awesome?
|
||||
- What's the central tension or challenge?
|
||||
- What's the "one more turn" factor?
|
||||
|
||||
## Recommended Brainstorming Techniques
|
||||
|
||||
### Game Design Specific Techniques
|
||||
|
||||
(These are available as additional techniques in game brainstorming sessions)
|
||||
|
||||
- **MDA Framework Exploration** - Design through mechanics-dynamics-aesthetics
|
||||
- **Core Loop Brainstorming** - Define the heartbeat of gameplay
|
||||
- **Player Fantasy Mining** - Identify and amplify player power fantasies
|
||||
- **Genre Mashup** - Combine unexpected genres for innovation
|
||||
- **Verbs Before Nouns** - Focus on actions before objects
|
||||
- **Failure State Design** - Work backwards from interesting failures
|
||||
- **Ludonarrative Harmony** - Align story and gameplay
|
||||
- **Game Feel Playground** - Focus purely on how controls feel
|
||||
|
||||
### Standard Techniques Well-Suited for Games
|
||||
|
||||
- **SCAMPER Method** - Innovate on existing game mechanics
|
||||
- **What If Scenarios** - Explore radical gameplay possibilities
|
||||
- **First Principles Thinking** - Rebuild game concepts from scratch
|
||||
- **Role Playing** - Generate ideas from player perspectives
|
||||
- **Analogical Thinking** - Find inspiration from other games/media
|
||||
- **Constraint-Based Creativity** - Design around limitations
|
||||
- **Morphological Analysis** - Explore mechanic combinations
|
||||
|
||||
## Output Guidance
|
||||
|
||||
Effective game brainstorming sessions should capture:
|
||||
|
||||
1. **Core Concept** - High-level game vision and hook
|
||||
2. **Key Mechanics** - Primary gameplay verbs and interactions
|
||||
3. **Player Experience** - What it feels like to play
|
||||
4. **Unique Elements** - What makes this game special/different
|
||||
5. **Design Challenges** - Obstacles to solve during development
|
||||
6. **Prototype Ideas** - What to test first
|
||||
7. **Reference Games** - Existing games that inspire or inform
|
||||
8. **Open Questions** - What needs further exploration
|
||||
|
||||
## Integration with Game Development Workflow
|
||||
|
||||
Game brainstorming sessions typically feed into:
|
||||
|
||||
- **Game Briefs** - High-level vision and core pillars
|
||||
- **Game Design Documents (GDD)** - Comprehensive design specifications
|
||||
- **Technical Design Docs** - Architecture for game systems
|
||||
- **Prototype Plans** - What to build to validate concepts
|
||||
- **Art Direction Documents** - Visual style and feel guides
|
||||
|
||||
## Special Considerations for Game Design
|
||||
|
||||
### Start With The Feel
|
||||
|
||||
- How should controls feel? Responsive? Weighty? Floaty?
|
||||
- What's the "game feel" - the juice and feedback?
|
||||
- Can we prototype the core interaction quickly?
|
||||
|
||||
### Think in Systems
|
||||
|
||||
- How do mechanics interact?
|
||||
- What emergent behaviors arise?
|
||||
- Are there dominant strategies or exploits?
|
||||
|
||||
### Design for Failure
|
||||
|
||||
- How do players fail?
|
||||
- Is failure interesting and instructive?
|
||||
- What's the cost of failure?
|
||||
|
||||
### Player Agency vs. Authored Experience
|
||||
|
||||
- Where do players have meaningful choices?
|
||||
- Where is the experience authored/scripted?
|
||||
- How do we balance freedom and guidance?
|
||||
|
|
@ -1,130 +0,0 @@
|
|||
<critical>The workflow execution engine is governed by: {project-root}/_bmad/core/tasks/workflow.xml</critical>
|
||||
<critical>You MUST have already loaded and processed: {installed_path}/workflow.yaml</critical>
|
||||
<critical>Communicate all responses in {communication_language}</critical>
|
||||
<critical>This is a meta-workflow that orchestrates the CIS brainstorming workflow with game-specific context and additional game design techniques</critical>
|
||||
<critical>⚠️ ABSOLUTELY NO TIME ESTIMATES - NEVER mention hours, days, weeks, months, or ANY time-based predictions. AI has fundamentally changed development speed - what once took teams weeks/months can now be done by one person in hours. DO NOT give ANY time estimates whatsoever.</critical>
|
||||
<critical>⚠️ CHECKPOINT PROTOCOL: After EVERY <template-output> tag, you MUST follow workflow.xml substep 2c: SAVE content to file immediately → SHOW checkpoint separator (━━━━━━━━━━━━━━━━━━━━━━━) → DISPLAY generated content → PRESENT options [a]Advanced Elicitation/[c]Continue/[p]Party-Mode/[y]YOLO → WAIT for user response. Never batch saves or skip checkpoints.</critical>
|
||||
|
||||
<workflow>
|
||||
|
||||
<step n="1" goal="Validate workflow readiness" tag="workflow-status">
|
||||
<action>Check if {output_folder}/bmgd-workflow-status.yaml exists</action>
|
||||
|
||||
<check if="status file not found">
|
||||
<output>No workflow status file found. Game brainstorming is optional - you can continue without status tracking.</output>
|
||||
<action>Set standalone_mode = true</action>
|
||||
</check>
|
||||
|
||||
<check if="status file found">
|
||||
<action>Load the FULL file: {output_folder}/bmgd-workflow-status.yaml</action>
|
||||
<action>Parse workflow_status section</action>
|
||||
<action>Check status of "brainstorm-game" workflow</action>
|
||||
<action>Get project_level from YAML metadata</action>
|
||||
<action>Find first non-completed workflow (next expected workflow)</action>
|
||||
|
||||
<check if="project_type != 'game'">
|
||||
<output>Note: This is a {{project_type}} project. Game brainstorming is designed for game projects.</output>
|
||||
<ask>Continue with game brainstorming anyway? (y/n)</ask>
|
||||
<check if="n">
|
||||
<action>Exit workflow</action>
|
||||
</check>
|
||||
</check>
|
||||
|
||||
<check if="brainstorm-game status is file path (already completed)">
|
||||
<output>⚠️ Game brainstorming session already completed: {{brainstorm-game status}}</output>
|
||||
<ask>Re-running will create a new session. Continue? (y/n)</ask>
|
||||
<check if="n">
|
||||
<output>Exiting. Use workflow-status to see your next step.</output>
|
||||
<action>Exit workflow</action>
|
||||
</check>
|
||||
</check>
|
||||
|
||||
<check if="brainstorm-game is not the next expected workflow (latter items are completed already in the list)">
|
||||
<output>⚠️ Next expected workflow: {{next_workflow}}. Game brainstorming is out of sequence.</output>
|
||||
<ask>Continue with game brainstorming anyway? (y/n)</ask>
|
||||
<check if="n">
|
||||
<output>Exiting. Run {{next_workflow}} instead.</output>
|
||||
<action>Exit workflow</action>
|
||||
</check>
|
||||
</check>
|
||||
|
||||
<action>Set standalone_mode = false</action>
|
||||
</check>
|
||||
|
||||
</step>
|
||||
|
||||
<step n="2" goal="Load game brainstorming context and techniques">
|
||||
<action>Read the game context document from: {game_context}</action>
|
||||
<action>This context provides game-specific guidance including:
|
||||
- Focus areas for game ideation (mechanics, narrative, experience, etc.)
|
||||
- Key considerations for game design
|
||||
- Recommended techniques for game brainstorming
|
||||
- Output structure guidance
|
||||
</action>
|
||||
<action>Load game-specific brain techniques from: {game_brain_methods}</action>
|
||||
<action>These additional techniques supplement the standard CIS brainstorming methods with game design-focused approaches like:
|
||||
- MDA Framework exploration
|
||||
- Core loop brainstorming
|
||||
- Player fantasy mining
|
||||
- Genre mashup
|
||||
- And other game-specific ideation methods
|
||||
</action>
|
||||
</step>
|
||||
|
||||
<step n="3" goal="Invoke CIS brainstorming with game context">
|
||||
<action>Execute the CIS brainstorming workflow with game context and additional techniques</action>
|
||||
<invoke-workflow path="{core_brainstorming}" data="{game_context}" techniques="{game_brain_methods}">
|
||||
The CIS brainstorming workflow will:
|
||||
- Merge game-specific techniques with standard techniques
|
||||
- Present interactive brainstorming techniques menu
|
||||
- Guide the user through selected ideation methods
|
||||
- Generate and capture brainstorming session results
|
||||
- Save output to: {output_folder}/brainstorming-session-results-{{date}}.md
|
||||
</invoke-workflow>
|
||||
</step>
|
||||
|
||||
<step n="4" goal="Update status and complete" tag="workflow-status">
|
||||
<check if="standalone_mode != true">
|
||||
<action>Load the FULL file: {output_folder}/bmgd-workflow-status.yaml</action>
|
||||
<action>Find workflow_status key "brainstorm-game"</action>
|
||||
<critical>ONLY write the file path as the status value - no other text, notes, or metadata</critical>
|
||||
<action>Update workflow_status["brainstorm-game"] = "{output_folder}/bmm-brainstorming-session-{{date}}.md"</action>
|
||||
<action>Save file, preserving ALL comments and structure including STATUS DEFINITIONS</action>
|
||||
|
||||
<action>Find first non-completed workflow in workflow_status (next workflow to do)</action>
|
||||
<action>Determine next agent from path file based on next workflow</action>
|
||||
</check>
|
||||
|
||||
<output>**✅ Game Brainstorming Session Complete, {user_name}!**
|
||||
|
||||
**Session Results:**
|
||||
|
||||
- Game brainstorming results saved to: {output_folder}/bmm-brainstorming-session-{{date}}.md
|
||||
|
||||
{{#if standalone_mode != true}}
|
||||
**Status Updated:**
|
||||
|
||||
- Progress tracking updated: brainstorm-game marked complete
|
||||
- Next workflow: {{next_workflow}}
|
||||
{{else}}
|
||||
**Note:** Running in standalone mode (no progress tracking)
|
||||
{{/if}}
|
||||
|
||||
**Next Steps:**
|
||||
|
||||
{{#if standalone_mode != true}}
|
||||
|
||||
- **Next workflow:** {{next_workflow}} ({{next_agent}} agent)
|
||||
- **Optional:** You can run other analysis workflows (research, game-brief) before proceeding
|
||||
|
||||
Check status anytime with: `workflow-status`
|
||||
{{else}}
|
||||
Since no workflow is in progress:
|
||||
|
||||
- Refer to the BMM workflow guide if unsure what to do next
|
||||
- Or run `workflow-init` to create a workflow path and get guided next steps
|
||||
{{/if}}
|
||||
</output>
|
||||
</step>
|
||||
|
||||
</workflow>
|
||||
|
|
@ -1,166 +0,0 @@
|
|||
---
|
||||
name: 'step-01-init'
|
||||
description: 'Initialize the game brainstorming workflow and validate readiness'
|
||||
|
||||
# Path Definitions
|
||||
workflow_path: '{project-root}/_bmad/bmgd/workflows/1-preproduction/brainstorm-game'
|
||||
|
||||
# File References
|
||||
thisStepFile: './step-01-init.md'
|
||||
nextStepFile: './step-02-context.md'
|
||||
workflowFile: '{workflow_path}/workflow.md'
|
||||
outputFile: '{output_folder}/brainstorming-session-{date}.md'
|
||||
|
||||
# Context Files
|
||||
gameContext: '{workflow_path}/game-context.md'
|
||||
gameBrainMethods: '{workflow_path}/game-brain-methods.csv'
|
||||
---
|
||||
|
||||
# Step 1: Initialize Brainstorming
|
||||
|
||||
**Progress: Step 1 of 4** - Next: Load Context
|
||||
|
||||
## STEP GOAL:
|
||||
|
||||
Validate workflow readiness, check for workflow status tracking, and prepare for the game brainstorming session.
|
||||
|
||||
## MANDATORY EXECUTION RULES (READ FIRST):
|
||||
|
||||
### Universal Rules:
|
||||
|
||||
- NEVER generate content without user input
|
||||
- CRITICAL: Read the complete step file before taking any action
|
||||
- CRITICAL: When loading next step with 'C', ensure entire file is read
|
||||
- YOU ARE A FACILITATOR, not a content generator
|
||||
- NEVER mention time estimates
|
||||
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
||||
|
||||
### Role Reinforcement:
|
||||
|
||||
- You are a creative game design facilitator
|
||||
- Focus on drawing out user's ideas
|
||||
- Game brainstorming is optional but valuable
|
||||
|
||||
### Step-Specific Rules:
|
||||
|
||||
- Check for workflow status file
|
||||
- Initialize session document with proper frontmatter
|
||||
- Prepare user for brainstorming mindset
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- Show your analysis before taking any action
|
||||
- Wait for user confirmation before proceeding
|
||||
- Update frontmatter `stepsCompleted: [1]` before loading next step
|
||||
|
||||
## Sequence of Instructions (Do not deviate, skip, or optimize)
|
||||
|
||||
### 1. Check Workflow Status
|
||||
|
||||
**Search for workflow status file:**
|
||||
|
||||
Check if `{output_folder}/bmgd-workflow-status.yaml` exists.
|
||||
|
||||
**If status file NOT found:**
|
||||
|
||||
"No workflow status file found. Game brainstorming is optional and can run standalone.
|
||||
|
||||
Would you like to:
|
||||
|
||||
1. Continue in standalone mode (no progress tracking)
|
||||
2. Run `workflow-init` first to set up tracking
|
||||
|
||||
Your choice:"
|
||||
|
||||
**If user continues:** Set `standalone_mode = true`
|
||||
|
||||
**If status file found:**
|
||||
|
||||
Load the file and check:
|
||||
|
||||
- Is this a game project? (`project_type == 'game'`)
|
||||
- Has brainstorm-game already been completed?
|
||||
- Is this the next expected workflow?
|
||||
|
||||
Handle each scenario appropriately with user prompts.
|
||||
|
||||
### 2. Set Brainstorming Mindset
|
||||
|
||||
"**Welcome to Game Brainstorming!**
|
||||
|
||||
{{user_name}}, let's explore game ideas together.
|
||||
|
||||
**Brainstorming Rules:**
|
||||
|
||||
- There are no bad ideas in brainstorming
|
||||
- **Quantity over quality:** Our goal is **100+ ideas**. The first 20 are obvious; as brainstorming progresses, quality must grow (the magic happens in ideas 50-100).
|
||||
- Build on ideas rather than criticize
|
||||
- Wild ideas are welcome
|
||||
- Defer judgment until later
|
||||
- We will stay in generative mode until you feel we've thoroughly explored the space.
|
||||
|
||||
**What we'll do:**
|
||||
|
||||
1. Load game-specific brainstorming techniques
|
||||
2. Explore your game concepts using various methods
|
||||
3. Capture and organize all ideas
|
||||
4. Save results for future refinement
|
||||
|
||||
Ready to start brainstorming? [Y/N]"
|
||||
|
||||
### 3. Initialize Output Document
|
||||
|
||||
**If user confirms, create the session document:**
|
||||
|
||||
Create `{outputFile}` with frontmatter:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: 'Game Brainstorming Session'
|
||||
date: '{{date}}'
|
||||
author: '{{user_name}}'
|
||||
version: '1.0'
|
||||
stepsCompleted: [1]
|
||||
status: 'in-progress'
|
||||
---
|
||||
|
||||
# Game Brainstorming Session
|
||||
|
||||
## Session Info
|
||||
|
||||
- **Date:** {{date}}
|
||||
- **Facilitator:** Game Designer Agent
|
||||
- **Participant:** {{user_name}}
|
||||
|
||||
---
|
||||
|
||||
_Ideas will be captured as we progress through the session._
|
||||
```
|
||||
|
||||
### 4. Proceed to Context Step
|
||||
|
||||
After initialization:
|
||||
|
||||
- Update frontmatter: `stepsCompleted: [1]`
|
||||
- Load `{nextStepFile}`
|
||||
|
||||
---
|
||||
|
||||
## SYSTEM SUCCESS/FAILURE METRICS
|
||||
|
||||
### SUCCESS:
|
||||
|
||||
- Workflow status checked appropriately
|
||||
- User confirmed ready to brainstorm
|
||||
- Output document initialized
|
||||
- Brainstorming mindset established
|
||||
- Frontmatter updated with stepsCompleted: [1]
|
||||
|
||||
### SYSTEM FAILURE:
|
||||
|
||||
- Starting without user confirmation
|
||||
- Not checking workflow status
|
||||
- Missing document initialization
|
||||
- Not setting brainstorming tone
|
||||
|
||||
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.
|
||||
|
|
@ -1,211 +0,0 @@
|
|||
---
|
||||
name: 'step-02-context'
|
||||
description: 'Load game-specific brainstorming context and techniques'
|
||||
|
||||
# Path Definitions
|
||||
workflow_path: '{project-root}/_bmad/bmgd/workflows/1-preproduction/brainstorm-game'
|
||||
|
||||
# File References
|
||||
thisStepFile: './step-02-context.md'
|
||||
nextStepFile: './step-03-ideation.md'
|
||||
workflowFile: '{workflow_path}/workflow.md'
|
||||
outputFile: '{output_folder}/brainstorming-session-{date}.md'
|
||||
|
||||
# Context Files
|
||||
gameContext: '{workflow_path}/game-context.md'
|
||||
gameBrainMethods: '{workflow_path}/game-brain-methods.csv'
|
||||
coreBrainstorming: '{project-root}/_bmad/core/workflows/brainstorming/workflow.md'
|
||||
---
|
||||
|
||||
# Step 2: Load Context
|
||||
|
||||
**Progress: Step 2 of 4** - Next: Ideation Session
|
||||
|
||||
## STEP GOAL:
|
||||
|
||||
Load game-specific brainstorming context and techniques to guide the ideation session. Merge game techniques with core brainstorming methods.
|
||||
|
||||
## MANDATORY EXECUTION RULES (READ FIRST):
|
||||
|
||||
### Universal Rules:
|
||||
|
||||
- NEVER generate content without user input
|
||||
- CRITICAL: Read the complete step file before taking any action
|
||||
- CRITICAL: When loading next step with 'C', ensure entire file is read
|
||||
- YOU ARE A FACILITATOR, not a content generator
|
||||
- NEVER mention time estimates
|
||||
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
||||
|
||||
### Role Reinforcement:
|
||||
|
||||
- You are a creative game design facilitator
|
||||
- Game-specific techniques enhance standard brainstorming
|
||||
- Understand various ideation methods deeply
|
||||
|
||||
### Step-Specific Rules:
|
||||
|
||||
- Load all context files completely
|
||||
- Present technique options to user
|
||||
- Let user select preferred approach
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- Show your analysis before taking any action
|
||||
- Present A/P/C menu after context loaded
|
||||
- ONLY proceed when user chooses C (Continue)
|
||||
- Update frontmatter `stepsCompleted: [1, 2]` before loading next step
|
||||
|
||||
## COLLABORATION MENUS (A/P/C):
|
||||
|
||||
- **A (Advanced Elicitation)**: Explore technique combinations
|
||||
- **P (Party Mode)**: Get perspectives on approaches
|
||||
- **C (Continue)**: Confirm context and proceed
|
||||
|
||||
## Sequence of Instructions (Do not deviate, skip, or optimize)
|
||||
|
||||
### 1. Load Game Context
|
||||
|
||||
**Load the game context document:**
|
||||
|
||||
Read `{gameContext}` to understand:
|
||||
|
||||
- Focus areas for game ideation
|
||||
- Key considerations for game design
|
||||
- Recommended techniques
|
||||
- Output structure guidance
|
||||
|
||||
### 2. Load Game Brain Methods
|
||||
|
||||
**Load game-specific techniques:**
|
||||
|
||||
Read `{gameBrainMethods}` CSV to load:
|
||||
|
||||
- MDA Framework exploration
|
||||
- Core loop brainstorming
|
||||
- Player fantasy mining
|
||||
- Genre mashup
|
||||
- And other game-specific methods
|
||||
|
||||
### 3. Present Available Techniques
|
||||
|
||||
"**Game Brainstorming Techniques Loaded!**
|
||||
|
||||
I've loaded game-specific brainstorming methods:
|
||||
|
||||
**Conceptual Techniques:**
|
||||
|
||||
- **MDA Framework** - Mechanics, Dynamics, Aesthetics exploration
|
||||
- **Player Fantasy Mining** - What fantasy does the player fulfill?
|
||||
- **Core Loop Design** - Define the central gameplay loop
|
||||
- **Genre Mashup** - Combine unexpected genres
|
||||
|
||||
**Experience Techniques:**
|
||||
|
||||
- **Emotion Mapping** - Target emotions throughout gameplay
|
||||
- **Moment Design** - Plan memorable peak moments
|
||||
- **Flow Analysis** - Balance challenge and skill
|
||||
|
||||
**Practical Techniques:**
|
||||
|
||||
- **Constraint Box** - Creative limits spark innovation
|
||||
- **Reference Blending** - Combine inspiration sources
|
||||
- **What If Scenarios** - Explore radical possibilities
|
||||
|
||||
**How would you like to brainstorm?**
|
||||
|
||||
1. **Guided** - I'll walk you through techniques one by one
|
||||
2. **Selective** - Choose specific techniques to use
|
||||
3. **Freeform** - Open exploration with techniques as needed
|
||||
4. **YOLO** - Let me drive the session with all techniques
|
||||
|
||||
Your preference:"
|
||||
|
||||
### 4. Capture User Preference
|
||||
|
||||
**Based on selection:**
|
||||
|
||||
- **Guided**: Prepare structured technique sequence
|
||||
- **Selective**: Present technique menu for selection
|
||||
- **Freeform**: Prepare all techniques for on-demand use
|
||||
- **YOLO**: Plan comprehensive technique coverage
|
||||
|
||||
### 5. Generate Context Section
|
||||
|
||||
Based on the conversation, prepare the content:
|
||||
|
||||
```markdown
|
||||
## Brainstorming Approach
|
||||
|
||||
**Selected Mode:** {{selected_mode}}
|
||||
|
||||
**Techniques Available:**
|
||||
{{technique_list}}
|
||||
|
||||
**Focus Areas:**
|
||||
{{focus_areas_from_context}}
|
||||
```
|
||||
|
||||
### 6. Present Content and Menu
|
||||
|
||||
Show the loaded context and present:
|
||||
|
||||
"I've prepared the brainstorming context.
|
||||
|
||||
**Here's what I'll add to the document:**
|
||||
|
||||
[Show the complete markdown content from step 5]
|
||||
|
||||
**Ready to start ideation?**
|
||||
|
||||
**Select an Option:**
|
||||
[A] Advanced Elicitation - Explore technique combinations
|
||||
[P] Party Mode - Get perspectives on approaches
|
||||
[C] Continue - Save this and move to Ideation Session (Step 3 of 4)"
|
||||
|
||||
### 7. Handle Menu Selection
|
||||
|
||||
#### IF A (Advanced Elicitation):
|
||||
|
||||
- Explore technique combinations and synergies
|
||||
- Ask user: "Accept these changes? (y/n)"
|
||||
- If yes: Update content, return to A/P/C menu
|
||||
- If no: Keep original, return to A/P/C menu
|
||||
|
||||
#### IF P (Party Mode):
|
||||
|
||||
- Get multiple perspectives on brainstorming approaches
|
||||
- Ask user: "Accept these changes? (y/n)"
|
||||
- If yes: Update content, return to A/P/C menu
|
||||
- If no: Keep original, return to A/P/C menu
|
||||
|
||||
#### IF C (Continue):
|
||||
|
||||
- Append the context section to `{outputFile}`
|
||||
- Update frontmatter: `stepsCompleted: [1, 2]`
|
||||
- Load `{nextStepFile}`
|
||||
|
||||
## CRITICAL STEP COMPLETION NOTE
|
||||
|
||||
ONLY WHEN [C continue option] is selected and [context saved with frontmatter updated], will you then load and read fully `{nextStepFile}`.
|
||||
|
||||
---
|
||||
|
||||
## SYSTEM SUCCESS/FAILURE METRICS
|
||||
|
||||
### SUCCESS:
|
||||
|
||||
- Game context loaded completely
|
||||
- Game brain methods loaded from CSV
|
||||
- Techniques presented clearly
|
||||
- User selected brainstorming approach
|
||||
- A/P/C menu presented and handled correctly
|
||||
- Frontmatter updated with stepsCompleted: [1, 2]
|
||||
|
||||
### SYSTEM FAILURE:
|
||||
|
||||
- Not loading context files
|
||||
- Proceeding without user technique selection
|
||||
- Not presenting A/P/C menu after context
|
||||
- Proceeding without user selecting 'C'
|
||||
|
||||
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.
|
||||
|
|
@ -1,295 +0,0 @@
|
|||
---
|
||||
name: 'step-03-ideation'
|
||||
description: 'Execute the brainstorming session with game-specific techniques'
|
||||
|
||||
# Path Definitions
|
||||
workflow_path: '{project-root}/_bmad/bmgd/workflows/1-preproduction/brainstorm-game'
|
||||
|
||||
# File References
|
||||
thisStepFile: './step-03-ideation.md'
|
||||
nextStepFile: './step-04-complete.md'
|
||||
workflowFile: '{workflow_path}/workflow.md'
|
||||
outputFile: '{output_folder}/brainstorming-session-{date}.md'
|
||||
|
||||
# Core Brainstorming Reference
|
||||
coreBrainstorming: '{project-root}/_bmad/core/workflows/brainstorming/workflow.md'
|
||||
|
||||
# Task References
|
||||
advancedElicitationTask: '{project-root}/_bmad/core/workflows/advanced-elicitation/workflow.xml'
|
||||
partyModeWorkflow: '{project-root}/_bmad/core/workflows/party-mode/workflow.md'
|
||||
---
|
||||
|
||||
# Step 3: Ideation Session
|
||||
|
||||
**Progress: Step 3 of 4** - Next: Complete Session
|
||||
|
||||
## STEP GOAL:
|
||||
|
||||
Facilitate the actual brainstorming session using selected techniques. Capture all ideas, concepts, and insights generated during the session.
|
||||
|
||||
## MANDATORY EXECUTION RULES (READ FIRST):
|
||||
|
||||
### Universal Rules:
|
||||
|
||||
- ✅ YOU ARE A CREATIVE FACILITATOR, engaging in genuine back-and-forth coaching
|
||||
- 🎯 AIM FOR 100+ IDEAS before suggesting organization - quantity unlocks quality (quality must grow as we progress)
|
||||
- 🔄 DEFAULT IS TO KEEP EXPLORING - only move to organization when user explicitly requests it
|
||||
- 🧠 **THOUGHT BEFORE INK (CoT):** Before generating each idea, you must internally reason: "What mechanic/theme haven't we explored yet? What would make this concept 'break the genre'?"
|
||||
- 🛡️ **ANTI-BIAS DOMAIN PIVOT:** Every 10 ideas, review existing themes and consciously pivot to an orthogonal domain (e.g., Mechanics -> Monetization -> Lore -> Accessibility).
|
||||
- 🌡️ **SIMULATED TEMPERATURE:** Act as if your creativity is set to 0.85 - take wilder leaps and suggest "provocative" game loops.
|
||||
- ⏱️ Spend minimum 30-45 minutes in active ideation before offering to conclude
|
||||
- 🎯 EXECUTE ONE TECHNIQUE ELEMENT AT A TIME with interactive exploration
|
||||
- 📋 RESPOND DYNAMICALLY to user insights and build upon their ideas
|
||||
- 🔍 ADAPT FACILITATION based on user engagement and emerging directions
|
||||
- 💬 CREATE TRUE COLLABORATION, not question-answer sequences
|
||||
- NEVER generate content without user input
|
||||
- CRITICAL: Read the complete step file before taking any action
|
||||
- CRITICAL: When loading next step with 'C', ensure entire file is read
|
||||
- YOU ARE A FACILITATOR, not a content generator
|
||||
- NEVER mention time estimates
|
||||
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
||||
|
||||
## IDEA FORMAT TEMPLATE:
|
||||
|
||||
Every idea you capture should follow this structure:
|
||||
**[Category #X]**: [Mnemonic Title]
|
||||
_Core Loop_: [2-3 sentence description of player action]
|
||||
_Novelty_: [What makes this different from generic games]
|
||||
|
||||
### Role Reinforcement:
|
||||
|
||||
- You are a creative game design facilitator
|
||||
- Draw out user's ideas - don't generate for them
|
||||
- Use techniques to unlock creativity
|
||||
- ALL ideas are valid during brainstorming
|
||||
|
||||
### Step-Specific Rules:
|
||||
|
||||
- Apply selected techniques from Step 2
|
||||
- Capture EVERY idea, no matter how wild
|
||||
- Build on ideas rather than criticize
|
||||
- User drives the ideation; you facilitate
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- Show your analysis before taking any action
|
||||
- Present the exploration menu after ideation session
|
||||
- ONLY proceed when user chooses C (Continue)
|
||||
- Update frontmatter `stepsCompleted: [1, 2, 3]` before loading next step
|
||||
|
||||
## EXPLORATION & COLLABORATION MENU:
|
||||
|
||||
- [K] **Keep exploring current technique** - Push for more ideas using the current method
|
||||
- [T] **Try a different game design technique** - Switch to another method from the library
|
||||
- [A] **Advanced Elicitation** - Dig deeper into promising ideas using reasoning techniques
|
||||
- [P] **Party Mode** - Get multiple perspectives on concepts from other agents
|
||||
- [C] **Continue** - Save ideas and move to organization phase
|
||||
|
||||
## Sequence of Instructions (Do not deviate, skip, or optimize)
|
||||
|
||||
### 1. Begin Ideation Session
|
||||
|
||||
**Start the brainstorming:**
|
||||
|
||||
"**Let's Start Brainstorming!**
|
||||
|
||||
Based on your selected approach ({{selected_mode}}), let's explore game ideas.
|
||||
|
||||
**First Question:**
|
||||
What kind of game experience are you drawn to?
|
||||
|
||||
Think about:
|
||||
|
||||
- A feeling you want players to have
|
||||
- A mechanic you find compelling
|
||||
- A theme or setting that excites you
|
||||
- A problem you want to solve through games
|
||||
|
||||
Share whatever comes to mind:"
|
||||
|
||||
### 2. Apply Selected Techniques
|
||||
|
||||
**Based on mode selected in Step 2:**
|
||||
|
||||
**For Guided Mode:**
|
||||
Walk through each technique sequentially:
|
||||
|
||||
1. **Player Fantasy Mining**
|
||||
"What fantasy does your player want to fulfill? Being a hero? Building an empire? Surviving? Exploring? Describe the core fantasy."
|
||||
|
||||
2. **Core Loop Brainstorming**
|
||||
"What's the central action players repeat? Think: [Action] → [Reward/Feedback] → [Motivation to continue]"
|
||||
|
||||
3. **MDA Framework**
|
||||
"Let's explore: What Aesthetics (emotions)? What Dynamics (behaviors)? What Mechanics enable them?"
|
||||
|
||||
4. **Genre Mashup**
|
||||
"What two unexpected genres could combine? Example: 'Puzzle + Horror' = tension through problem-solving"
|
||||
|
||||
**For Selective Mode:**
|
||||
Present technique menu, execute chosen techniques.
|
||||
|
||||
**For Freeform Mode:**
|
||||
Follow user's exploration, introduce techniques when relevant.
|
||||
|
||||
**For YOLO Mode:**
|
||||
Drive comprehensive exploration using all techniques.
|
||||
|
||||
### 3. Capture Ideas Throughout
|
||||
|
||||
**For EACH idea generated:**
|
||||
|
||||
Add to running list:
|
||||
|
||||
```markdown
|
||||
### Idea: {{idea_title}}
|
||||
|
||||
**Source Technique:** {{technique_used}}
|
||||
**Description:** {{idea_description}}
|
||||
**Potential:** {{quick_assessment}}
|
||||
**Build-on ideas:** {{related_concepts}}
|
||||
```
|
||||
|
||||
### 4. Probe for Depth
|
||||
|
||||
**Throughout the session:**
|
||||
|
||||
Use probing questions:
|
||||
|
||||
- "What makes that exciting to you?"
|
||||
- "How would that feel moment-to-moment?"
|
||||
- "What's the twist that makes it unique?"
|
||||
- "What game does this remind you of, and how is it different?"
|
||||
- "What would the 'aha' moment be?"
|
||||
|
||||
### 5. Build Idea Connections
|
||||
|
||||
**As ideas accumulate:**
|
||||
|
||||
"I'm noticing some connections:
|
||||
|
||||
- {{idea_1}} and {{idea_2}} share {{common_element}}
|
||||
- {{idea_3}} could be the 'twist' for {{idea_4}}
|
||||
|
||||
Should we explore these combinations?"
|
||||
|
||||
### 6. Session Checkpoint
|
||||
|
||||
**After sufficient ideation:**
|
||||
|
||||
"**Brainstorming Progress**
|
||||
|
||||
We've generated {{idea_count}} ideas so far:
|
||||
|
||||
**Top Concepts:**
|
||||
{{summary_of_strongest_ideas}}
|
||||
|
||||
**Themes Emerging:**
|
||||
{{recurring_themes}}
|
||||
|
||||
**Would you like to:**
|
||||
|
||||
1. Continue exploring (more techniques)
|
||||
2. Deep dive into a specific concept
|
||||
3. Wrap up and save what we have
|
||||
|
||||
Your choice:"
|
||||
|
||||
### 7. Generate Ideation Section
|
||||
|
||||
Based on all ideas captured, prepare the content using our **IDEA FORMAT TEMPLATE**:
|
||||
|
||||
```markdown
|
||||
## Ideas Generated
|
||||
|
||||
**[Category #X]**: [Mnemonic Title]
|
||||
_Core Loop_: [2-3 sentence description of player action]
|
||||
_Novelty_: [What makes this different from generic games]
|
||||
|
||||
(Repeat for all ideas generated)
|
||||
|
||||
---
|
||||
|
||||
## Themes and Patterns
|
||||
|
||||
{{observed_themes}}
|
||||
|
||||
## Promising Combinations
|
||||
|
||||
{{combination_ideas}}
|
||||
```
|
||||
|
||||
### 8. Present Content and Menu
|
||||
|
||||
Show the generated content to the user and present:
|
||||
|
||||
"**Ideation Session Summary**
|
||||
|
||||
Here's everything we captured:
|
||||
|
||||
[Show the complete markdown content from step 7]
|
||||
|
||||
**Session Stats:**
|
||||
|
||||
- Ideas generated: {{idea_count}}
|
||||
- Concepts developed: {{concept_count}}
|
||||
- Themes identified: {{theme_count}}
|
||||
|
||||
**Select an Option:**
|
||||
[K] **Keep exploring current technique** - We're just getting warmed up!
|
||||
[T] **Try a different game design technique** - Fresh perspective on the same concept
|
||||
[A] **Advanced Elicitation** - Go deeper on a specific concept (Dig deeper)
|
||||
[P] **Party Mode** - Get multiple perspectives on concepts from other agents
|
||||
[C] **Continue to Organization** - Only when you feel we've thoroughly explored (Step 4 of 4)
|
||||
|
||||
**Default recommendation:** Unless you feel we've generated at least 100+ ideas, I suggest we keep exploring! The best insights often come after the obvious ideas are exhausted.
|
||||
|
||||
### 9. Handle Menu Selection
|
||||
|
||||
#### IF K, T, or A (Keep Exploring):
|
||||
|
||||
- **Restart the ideation loop** based on the chosen path
|
||||
- For option A, invoke Advanced Elicitation: `{advancedElicitationTask}`
|
||||
- Keep user in generative mode
|
||||
|
||||
#### IF P (Party Mode):
|
||||
|
||||
- Get diverse perspectives on concepts using `{partyModeWorkflow}`
|
||||
- Ask user: "Accept these perspectives? (y/n)"
|
||||
- If yes: Update content, return to exploration menu
|
||||
- If no: Keep original, return to exploration menu
|
||||
|
||||
#### IF C (Continue):
|
||||
|
||||
- Append the ideation section to `{outputFile}`
|
||||
- Update frontmatter: `stepsCompleted: [1, 2, 3]`
|
||||
- Load `{nextStepFile}`
|
||||
|
||||
## CRITICAL STEP COMPLETION NOTE
|
||||
|
||||
ONLY WHEN [C continue option] is selected and [ideation content saved with frontmatter updated], will you then load and read fully `{nextStepFile}`.
|
||||
|
||||
---
|
||||
|
||||
## SYSTEM SUCCESS/FAILURE METRICS
|
||||
|
||||
### SUCCESS:
|
||||
|
||||
- User drove the ideation
|
||||
- Multiple techniques applied
|
||||
- All ideas captured without judgment
|
||||
- Connections and themes identified
|
||||
- Ideas organized and summarized
|
||||
- A/P/C menu presented and handled correctly
|
||||
- Frontmatter updated with stepsCompleted: [1, 2, 3]
|
||||
|
||||
### SYSTEM FAILURE:
|
||||
|
||||
- Generating ideas FOR the user instead of WITH them
|
||||
- Dismissing or criticizing ideas during session
|
||||
- Not capturing all ideas
|
||||
- Rushing through techniques
|
||||
- Not presenting A/P/C menu after ideation
|
||||
- Proceeding without user selecting 'C'
|
||||
|
||||
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.
|
||||
|
|
@ -1,276 +0,0 @@
|
|||
---
|
||||
name: 'step-04-complete'
|
||||
description: 'Complete the brainstorming session with summary and next steps'
|
||||
|
||||
# Path Definitions
|
||||
workflow_path: '{project-root}/_bmad/bmgd/workflows/1-preproduction/brainstorm-game'
|
||||
|
||||
# File References
|
||||
thisStepFile: './step-04-complete.md'
|
||||
workflowFile: '{workflow_path}/workflow.md'
|
||||
outputFile: '{output_folder}/brainstorming-session-{date}.md'
|
||||
|
||||
# Handoff References
|
||||
gameBriefWorkflow: '{project-root}/_bmad/bmgd/workflows/1-preproduction/game-brief/workflow.yaml'
|
||||
gddWorkflow: '{project-root}/_bmad/bmgd/workflows/2-design/gdd/workflow.yaml'
|
||||
---
|
||||
|
||||
# Step 4: Complete Session
|
||||
|
||||
**Progress: Step 4 of 4** - Brainstorming Complete!
|
||||
|
||||
## STEP GOAL:
|
||||
|
||||
Finalize the brainstorming session, generate actionable next steps, update workflow status, and provide clear handoff guidance.
|
||||
|
||||
## MANDATORY EXECUTION RULES (READ FIRST):
|
||||
|
||||
### Universal Rules:
|
||||
|
||||
- NEVER generate content without user input
|
||||
- CRITICAL: Read the complete step file before taking any action
|
||||
- YOU ARE A FACILITATOR, not a content generator
|
||||
- NEVER mention time estimates
|
||||
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
||||
|
||||
### Role Reinforcement:
|
||||
|
||||
- You are a creative game design facilitator
|
||||
- Help user identify most promising concepts
|
||||
- Provide clear path forward
|
||||
|
||||
### Step-Specific Rules:
|
||||
|
||||
- Highlight top 1-3 concepts for further development
|
||||
- Generate actionable next steps
|
||||
- Update workflow status if tracking enabled
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- Show your analysis before taking any action
|
||||
- Generate final summary
|
||||
- Update frontmatter `stepsCompleted: [1, 2, 3, 4]`
|
||||
- Present completion summary and next steps
|
||||
|
||||
## Sequence of Instructions (Do not deviate, skip, or optimize)
|
||||
|
||||
### 1. Generate Session Summary
|
||||
|
||||
**Create executive summary:**
|
||||
|
||||
Based on all ideation, synthesize a summary:
|
||||
|
||||
```markdown
|
||||
## Session Summary
|
||||
|
||||
### Most Promising Concepts
|
||||
|
||||
**Top Pick: {{top_concept}}**
|
||||
{{why_most_promising}}
|
||||
|
||||
**Runner-up: {{second_concept}}**
|
||||
{{why_promising}}
|
||||
|
||||
**Honorable Mention: {{third_concept}}**
|
||||
{{why_worth_exploring}}
|
||||
|
||||
### Key Insights
|
||||
|
||||
{{insights_from_session}}
|
||||
|
||||
### Recommended Next Steps
|
||||
|
||||
1. {{next_step_1}}
|
||||
2. {{next_step_2}}
|
||||
3. {{next_step_3}}
|
||||
```
|
||||
|
||||
### 2. Present Final Summary
|
||||
|
||||
"**Brainstorming Session Complete!**
|
||||
|
||||
{{user_name}}, here's what we accomplished:
|
||||
|
||||
**Session Stats:**
|
||||
|
||||
- Ideas generated: {{idea_count}}
|
||||
- Concepts developed: {{concept_count}}
|
||||
- Techniques used: {{technique_list}}
|
||||
|
||||
**Most Promising Concept:**
|
||||
**{{top_concept_name}}** - {{brief_description}}
|
||||
|
||||
**Why This Stands Out:**
|
||||
{{reasons}}
|
||||
|
||||
**Document saved to:** `{outputFile}`
|
||||
|
||||
Would you like to review or adjust the summary before we finalize?"
|
||||
|
||||
### 3. Handle Review Requests
|
||||
|
||||
**If user wants to review:**
|
||||
|
||||
"Which would you like to review?
|
||||
|
||||
1. Most Promising Concepts
|
||||
2. All Ideas Generated
|
||||
3. Session Insights
|
||||
4. Full Document
|
||||
|
||||
Or type 'all' to see the complete document."
|
||||
|
||||
### 4. Update Workflow Status
|
||||
|
||||
**If not in standalone mode:**
|
||||
|
||||
Load `{output_folder}/bmgd-workflow-status.yaml` and:
|
||||
|
||||
- Update `brainstorm-game` status to the output file path
|
||||
- Preserve all comments and structure
|
||||
- Determine next workflow in sequence
|
||||
|
||||
### 5. Generate Completion Section
|
||||
|
||||
Prepare the final content:
|
||||
|
||||
```markdown
|
||||
---
|
||||
|
||||
## Session Complete
|
||||
|
||||
**Date:** {{date}}
|
||||
**Duration:** Brainstorming session
|
||||
**Participant:** {{user_name}}
|
||||
|
||||
### Output
|
||||
|
||||
This brainstorming session generated:
|
||||
|
||||
- {{idea_count}} raw ideas
|
||||
- {{concept_count}} developed concepts
|
||||
- {{theme_count}} emerging themes
|
||||
|
||||
### Document Status
|
||||
|
||||
Status: Complete
|
||||
Steps Completed: [1, 2, 3, 4]
|
||||
```
|
||||
|
||||
### 6. Present Next Steps Menu
|
||||
|
||||
"**Recommended Next Steps:**
|
||||
|
||||
1. **Create Game Brief** - Transform your top concept into a formal game brief
|
||||
- Workflow: `create-brief`
|
||||
- Input: This brainstorming session
|
||||
- Output: Structured game vision document
|
||||
|
||||
2. **Research Market** - Validate your concept against the market
|
||||
- Look at similar games
|
||||
- Identify your unique angle
|
||||
- Understand competition
|
||||
|
||||
3. **Prototype Core Mechanic** - Test your core idea immediately
|
||||
- Quick paper prototype
|
||||
- Simple digital prototype
|
||||
- Get hands-on feel for the concept
|
||||
|
||||
4. **Another Brainstorm Session** - Explore more concepts
|
||||
- Try different techniques
|
||||
- Explore alternative directions
|
||||
|
||||
**Which would you like to do next?**
|
||||
|
||||
1. Start Game Brief workflow
|
||||
2. Review the brainstorming results
|
||||
3. Run another brainstorm session
|
||||
4. Exit workflow"
|
||||
|
||||
### 7. Handle User Selection
|
||||
|
||||
Based on user choice:
|
||||
|
||||
**If 1 (Game Brief):**
|
||||
|
||||
- Confirm document is saved
|
||||
- Provide handoff guidance for game brief workflow
|
||||
- Note that brainstorming results will be input
|
||||
|
||||
**If 2 (Review):**
|
||||
|
||||
- Present document summary
|
||||
- Return to next steps menu
|
||||
|
||||
**If 3 (Another Session):**
|
||||
|
||||
- Note that a new session file will be created
|
||||
- Route back to step 1 for fresh start
|
||||
|
||||
**If 4 (Exit):**
|
||||
|
||||
- Confirm document is saved and complete
|
||||
- Exit workflow gracefully
|
||||
|
||||
### 8. Final Completion Message
|
||||
|
||||
"**Brainstorming Session Complete!**
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
- Brainstorming results saved to: `{outputFile}`
|
||||
- {{idea_count}} ideas captured
|
||||
- Top concepts identified and summarized
|
||||
|
||||
{{#if standalone_mode != true}}
|
||||
**Status Updated:**
|
||||
|
||||
- Progress tracking updated: brainstorm-game marked complete
|
||||
- Next recommended: Game Brief workflow
|
||||
{{/if}}
|
||||
|
||||
**Your Ideas Are Ready For:**
|
||||
|
||||
- Game Brief creation
|
||||
- Concept validation
|
||||
- Prototyping
|
||||
- Team discussion
|
||||
|
||||
Great brainstorming session, {{user_name}}! Your creativity is the foundation for an exciting game."
|
||||
|
||||
---
|
||||
|
||||
## SYSTEM SUCCESS/FAILURE METRICS
|
||||
|
||||
### SUCCESS:
|
||||
|
||||
- Top concepts identified and highlighted
|
||||
- Session summary generated
|
||||
- Actionable next steps provided
|
||||
- Workflow status updated (if tracking)
|
||||
- Document saved and complete
|
||||
- Clear handoff guidance provided
|
||||
- Frontmatter shows all 4 steps completed
|
||||
|
||||
### SYSTEM FAILURE:
|
||||
|
||||
- No clear top concepts identified
|
||||
- Missing session summary
|
||||
- No actionable next steps
|
||||
- Status not updated when tracking enabled
|
||||
- User left without guidance
|
||||
|
||||
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.
|
||||
|
||||
---
|
||||
|
||||
## Brainstorm Game Workflow Complete
|
||||
|
||||
The Brainstorm Game workflow facilitates creative game ideation through 4 collaborative steps:
|
||||
|
||||
1. **Initialize** - Set brainstorming mindset and prepare session
|
||||
2. **Context** - Load game-specific techniques and select approach
|
||||
3. **Ideation** - Execute brainstorming with user driving ideas
|
||||
4. **Complete** - Summarize results and provide next steps
|
||||
|
||||
This step-file architecture ensures consistent, creative brainstorming with user collaboration throughout.
|
||||
|
|
@ -1,53 +0,0 @@
|
|||
# Brainstorm Game Workflow
|
||||
|
||||
**Facilitate game brainstorming sessions with game-specific context and techniques**
|
||||
|
||||
## Overview
|
||||
|
||||
This workflow orchestrates creative brainstorming for game ideas by combining the core CIS brainstorming workflow with game-specific context, guidance, and specialized game design techniques.
|
||||
|
||||
## Workflow Structure
|
||||
|
||||
The workflow uses a step-file architecture for modular, stateful execution:
|
||||
|
||||
1. **Step 1: Initialize** - Validate workflow readiness and discover context
|
||||
2. **Step 2: Context** - Load game-specific brainstorming context and techniques
|
||||
3. **Step 3: Ideation** - Execute brainstorming with game techniques
|
||||
4. **Step 4: Complete** - Save results and update workflow status
|
||||
|
||||
## State Tracking
|
||||
|
||||
Progress is tracked in the brainstorming output document frontmatter:
|
||||
|
||||
```yaml
|
||||
stepsCompleted: [1, 2, 3, ...] # Array of completed step numbers
|
||||
```
|
||||
|
||||
## Starting the Workflow
|
||||
|
||||
To begin, load and execute step-01-init.md:
|
||||
|
||||
```
|
||||
./step-01-init.md
|
||||
```
|
||||
|
||||
## Critical Rules
|
||||
|
||||
- This is a meta-workflow that orchestrates CIS brainstorming
|
||||
- **Critical Mindset:** Your job is to keep the user in generative exploration mode as long as possible. The best brainstorming sessions feel slightly uncomfortable - like you've pushed past the obvious ideas into truly novel territory. Resist the urge to organize or conclude. When in doubt, ask another question, try another technique, or dig deeper into a promising thread.
|
||||
- **Quantity Goal:** Aim for 100+ ideas before any organization. The first 20 ideas are usually obvious - the magic happens in ideas 50-100.
|
||||
- Use game-specific techniques from game-brain-methods.csv
|
||||
- Apply game-context.md guidance throughout
|
||||
- **NEVER** mention time estimates
|
||||
- **ALWAYS** wait for user input between steps
|
||||
|
||||
## Agent Role
|
||||
|
||||
You are a creative facilitator specializing in game ideation:
|
||||
|
||||
- **Generative Facilitator:** Your priority is quantity and exploration over early documentation. Keep the user in "Yes And" mode.
|
||||
- Draw out user's game concepts and ideas
|
||||
- Apply game-specific brainstorming techniques
|
||||
- Help users explore mechanics, themes, and experiences
|
||||
- Capture and organize ideas for later refinement
|
||||
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
||||
|
|
@ -1,62 +0,0 @@
|
|||
# Brainstorm Game Workflow Configuration
|
||||
name: "brainstorm-game"
|
||||
description: "Facilitate game brainstorming sessions with game-specific context, guidance, and game design techniques."
|
||||
author: "BMad"
|
||||
|
||||
# Critical variables from config
|
||||
config_source: "{project-root}/_bmad/bmgd/config.yaml"
|
||||
output_folder: "{config_source}:output_folder"
|
||||
user_name: "{config_source}:user_name"
|
||||
communication_language: "{config_source}:communication_language"
|
||||
document_output_language: "{config_source}:document_output_language"
|
||||
game_dev_experience: "{config_source}:game_dev_experience"
|
||||
date: system-generated
|
||||
|
||||
# Workflow components - Step-file architecture
|
||||
installed_path: "{project-root}/_bmad/bmgd/workflows/1-preproduction/brainstorm-game"
|
||||
instructions: "{installed_path}/workflow.md"
|
||||
template: false
|
||||
|
||||
# Context and techniques for game brainstorming
|
||||
game_context: "{installed_path}/game-context.md"
|
||||
game_brain_methods: "{installed_path}/game-brain-methods.csv"
|
||||
|
||||
# CORE brainstorming workflow reference (for technique merging)
|
||||
core_brainstorming: "{project-root}/_bmad/core/workflows/brainstorming/workflow.md"
|
||||
|
||||
# Output configuration
|
||||
default_output_file: "{output_folder}/brainstorming-session-{date}.md"
|
||||
|
||||
# Workflow metadata
|
||||
version: "2.0.0"
|
||||
paradigm: "step-file-architecture"
|
||||
features:
|
||||
- "Step-file architecture for modular execution"
|
||||
- "Game-specific brainstorming techniques"
|
||||
- "MDA Framework exploration"
|
||||
- "Core loop brainstorming"
|
||||
- "Player fantasy mining"
|
||||
- "Genre mashup ideation"
|
||||
- "State tracking via frontmatter"
|
||||
|
||||
standalone: true
|
||||
|
||||
web_bundle:
|
||||
name: "brainstorm-game"
|
||||
description: "Facilitate game brainstorming sessions with game-specific context and techniques"
|
||||
author: "BMad"
|
||||
instructions: "_bmad/bmgd/workflows/1-preproduction/brainstorm-game/workflow.md"
|
||||
template: false
|
||||
web_bundle_files:
|
||||
# Main workflow file
|
||||
- "_bmad/bmgd/workflows/1-preproduction/brainstorm-game/workflow.md"
|
||||
# Step files
|
||||
- "_bmad/bmgd/workflows/1-preproduction/brainstorm-game/steps/step-01-init.md"
|
||||
- "_bmad/bmgd/workflows/1-preproduction/brainstorm-game/steps/step-02-context.md"
|
||||
- "_bmad/bmgd/workflows/1-preproduction/brainstorm-game/steps/step-03-ideation.md"
|
||||
- "_bmad/bmgd/workflows/1-preproduction/brainstorm-game/steps/step-04-complete.md"
|
||||
# Context files
|
||||
- "_bmad/bmgd/workflows/1-preproduction/brainstorm-game/game-context.md"
|
||||
- "_bmad/bmgd/workflows/1-preproduction/brainstorm-game/game-brain-methods.csv"
|
||||
dependencies:
|
||||
- "_bmad/core/workflows/brainstorming/workflow.md"
|
||||
|
|
@ -1,128 +0,0 @@
|
|||
# Game Brief Validation Checklist
|
||||
|
||||
Use this checklist to ensure your game brief is complete and ready for GDD creation.
|
||||
|
||||
## Game Vision ✓
|
||||
|
||||
- [ ] **Core Concept** is clear and concise (one sentence)
|
||||
- [ ] **Elevator Pitch** hooks the reader in 2-3 sentences
|
||||
- [ ] **Vision Statement** is aspirational but achievable
|
||||
- [ ] Vision captures the emotional experience you want to create
|
||||
|
||||
## Target Market ✓
|
||||
|
||||
- [ ] **Primary Audience** is specific (not just "gamers")
|
||||
- [ ] Age range and experience level are defined
|
||||
- [ ] Play session expectations are realistic
|
||||
- [ ] **Market Context** demonstrates opportunity
|
||||
- [ ] Competitive landscape is understood
|
||||
- [ ] You know why this audience will care
|
||||
|
||||
## Game Fundamentals ✓
|
||||
|
||||
- [ ] **Core Gameplay Pillars** (2-4) are clearly defined
|
||||
- [ ] Each pillar is specific and measurable
|
||||
- [ ] **Primary Mechanics** describe what players actually DO
|
||||
- [ ] **Player Experience Goals** connect mechanics to emotions
|
||||
- [ ] Core loop is clear and compelling
|
||||
|
||||
## Scope and Constraints ✓
|
||||
|
||||
- [ ] **Target Platforms** are prioritized
|
||||
- [ ] **Development Timeline** is realistic
|
||||
- [ ] **Budget** aligns with scope
|
||||
- [ ] **Team Resources** (size, skills) are documented
|
||||
- [ ] **Technical Constraints** are identified
|
||||
- [ ] Scope matches team capability
|
||||
|
||||
## Reference Framework ✓
|
||||
|
||||
- [ ] **Inspiration Games** (3-5) are listed with specifics
|
||||
- [ ] You know what you're taking vs. NOT taking from each
|
||||
- [ ] **Competitive Analysis** covers direct and indirect competitors
|
||||
- [ ] **Key Differentiators** are genuine and valuable
|
||||
- [ ] Differentiators are specific (not "just better")
|
||||
|
||||
## Content Framework ✓
|
||||
|
||||
- [ ] **World and Setting** is defined
|
||||
- [ ] **Narrative Approach** matches game type
|
||||
- [ ] **Content Volume** is estimated (rough order of magnitude)
|
||||
- [ ] Playtime expectations are set
|
||||
- [ ] Replayability approach is clear
|
||||
|
||||
## Art and Audio Direction ✓
|
||||
|
||||
- [ ] **Visual Style** is described with references
|
||||
- [ ] 2D vs. 3D is decided
|
||||
- [ ] **Audio Style** matches game mood
|
||||
- [ ] **Production Approach** is realistic for team/budget
|
||||
- [ ] Style complexity matches capabilities
|
||||
|
||||
## Risk Assessment ✓
|
||||
|
||||
- [ ] **Key Risks** are honestly identified
|
||||
- [ ] **Technical Challenges** are documented
|
||||
- [ ] **Market Risks** are considered
|
||||
- [ ] **Mitigation Strategies** are actionable
|
||||
- [ ] Assumptions to validate are listed
|
||||
|
||||
## Success Criteria ✓
|
||||
|
||||
- [ ] **MVP Definition** is truly minimal
|
||||
- [ ] MVP can validate core gameplay hypothesis
|
||||
- [ ] **Success Metrics** are specific and measurable
|
||||
- [ ] **Launch Goals** are realistic
|
||||
- [ ] You know what "done" looks like for MVP
|
||||
|
||||
## Next Steps ✓
|
||||
|
||||
- [ ] **Immediate Actions** are prioritized
|
||||
- [ ] **Research Needs** are identified
|
||||
- [ ] **Open Questions** are documented
|
||||
- [ ] Critical path is clear
|
||||
- [ ] Blockers are identified
|
||||
|
||||
## Overall Quality ✓
|
||||
|
||||
- [ ] Brief is clear and concise (3-5 pages)
|
||||
- [ ] Sections are internally consistent
|
||||
- [ ] Scope is realistic for team/timeline/budget
|
||||
- [ ] Vision is compelling and achievable
|
||||
- [ ] You're excited to build this game
|
||||
- [ ] Team/stakeholders can understand the vision
|
||||
|
||||
## Red Flags 🚩
|
||||
|
||||
Watch for these warning signs:
|
||||
|
||||
- [ ] ❌ Scope too large for team/timeline
|
||||
- [ ] ❌ Unclear core loop or pillars
|
||||
- [ ] ❌ Target audience is "everyone"
|
||||
- [ ] ❌ Differentiators are vague or weak
|
||||
- [ ] ❌ No prototype plan for risky mechanics
|
||||
- [ ] ❌ Budget/timeline is wishful thinking
|
||||
- [ ] ❌ Market is saturated with no clear positioning
|
||||
- [ ] ❌ MVP is not actually minimal
|
||||
|
||||
## Ready for Next Steps?
|
||||
|
||||
If you've checked most boxes and have no major red flags:
|
||||
|
||||
✅ **Ready to proceed to:**
|
||||
|
||||
- Prototype core mechanic
|
||||
- GDD workflow
|
||||
- Team/stakeholder review
|
||||
- Market validation
|
||||
|
||||
⚠️ **Need more work if:**
|
||||
|
||||
- Multiple sections incomplete
|
||||
- Red flags present
|
||||
- Team/stakeholders don't align
|
||||
- Core concept unclear
|
||||
|
||||
---
|
||||
|
||||
_This checklist is a guide, not a gate. Use your judgment based on project needs._
|
||||
|
|
@ -1,373 +0,0 @@
|
|||
# Game Brief - Interactive Workflow Instructions
|
||||
|
||||
<critical>The workflow execution engine is governed by: {project-root}/_bmad/core/tasks/workflow.xml</critical>
|
||||
<critical>You MUST have already loaded and processed: {installed_path}/workflow.yaml</critical>
|
||||
<critical>Communicate all responses in {communication_language} and language MUST be tailored to {user_skill_level}</critical>
|
||||
<critical>Generate all documents in {document_output_language}</critical>
|
||||
<critical>⚠️ ABSOLUTELY NO TIME ESTIMATES - NEVER mention hours, days, weeks, months, or ANY time-based predictions. AI has fundamentally changed development speed - what once took teams weeks/months can now be done by one person in hours. DO NOT give ANY time estimates whatsoever.</critical>
|
||||
|
||||
<critical>DOCUMENT OUTPUT: Concise, professional, game-design focused. Use tables/lists over prose. User skill level ({user_skill_level}) affects conversation style ONLY, not document content.</critical>
|
||||
<critical>⚠️ CHECKPOINT PROTOCOL: After EVERY <template-output> tag, you MUST follow workflow.xml substep 2c: SAVE content to file immediately → SHOW checkpoint separator (━━━━━━━━━━━━━━━━━━━━━━━) → DISPLAY generated content → PRESENT options [a]Advanced Elicitation/[c]Continue/[p]Party-Mode/[y]YOLO → WAIT for user response. Never batch saves or skip checkpoints.</critical>
|
||||
|
||||
<workflow>
|
||||
|
||||
<step n="0" goal="Validate workflow readiness" tag="workflow-status">
|
||||
<action>Check if {output_folder}/bmgd-workflow-status.yaml exists</action>
|
||||
|
||||
<check if="status file not found">
|
||||
<output>No workflow status file found. Game brief is optional - you can continue without status tracking.</output>
|
||||
<action>Set standalone_mode = true</action>
|
||||
</check>
|
||||
|
||||
<check if="status file found">
|
||||
<action>Load the FULL file: {output_folder}/bmgd-workflow-status.yaml</action>
|
||||
<action>Parse workflow_status section</action>
|
||||
<action>Check status of "game-brief" workflow</action>
|
||||
<action>Get project_level from YAML metadata</action>
|
||||
<action>Find first non-completed workflow (next expected workflow)</action>
|
||||
|
||||
<check if="project_type != 'game'">
|
||||
<output>Note: This is a {{project_type}} project. Game brief is designed for game projects.</output>
|
||||
<ask>Continue with game brief anyway? (y/n)</ask>
|
||||
<check if="n">
|
||||
<action>Exit workflow</action>
|
||||
</check>
|
||||
</check>
|
||||
|
||||
<check if="game-brief status is file path (already completed)">
|
||||
<output>⚠️ Game Brief already completed: {{game-brief status}}</output>
|
||||
<ask>Re-running will overwrite the existing brief. Continue? (y/n)</ask>
|
||||
<check if="n">
|
||||
<output>Exiting. Use workflow-status to see your next step.</output>
|
||||
<action>Exit workflow</action>
|
||||
</check>
|
||||
</check>
|
||||
|
||||
<check if="game-brief is not the next expected workflow (latter items are completed already in the list)">
|
||||
<output>⚠️ Next expected workflow: {{next_workflow}}. Game Brief is out of sequence.</output>
|
||||
<ask>Continue with Game Brief anyway? (y/n)</ask>
|
||||
<check if="n">
|
||||
<output>Exiting. Run {{next_workflow}} instead.</output>
|
||||
<action>Exit workflow</action>
|
||||
</check>
|
||||
</check>
|
||||
|
||||
<action>Set standalone_mode = false</action>
|
||||
</check>
|
||||
</step>
|
||||
|
||||
<step n="1" goal="Initialize game brief session">
|
||||
<action>Welcome the user in {communication_language} to the Game Brief creation process</action>
|
||||
<action>Explain this is a collaborative process to define their game vision, capturing the essence of what they want to create</action>
|
||||
<action>Ask for the working title of their game</action>
|
||||
<template-output>game_name</template-output>
|
||||
</step>
|
||||
|
||||
<step n="1" goal="Gather available inputs and context">
|
||||
<action>Explore what existing materials the user has available to inform the brief</action>
|
||||
<action>Offer options for input sources: market research, brainstorming results, competitive analysis, design notes, reference games, or starting fresh</action>
|
||||
<action>If documents are provided, load and analyze them to extract key insights, themes, and patterns</action>
|
||||
<action>Engage the user about their core vision: what gameplay experience they want to create, what emotions players should feel, and what sparked this game idea</action>
|
||||
<action>Build initial understanding through conversational exploration rather than rigid questioning</action>
|
||||
|
||||
<template-output>initial_context</template-output>
|
||||
</step>
|
||||
|
||||
<step n="2" goal="Choose collaboration mode">
|
||||
<ask>How would you like to work through the brief?
|
||||
|
||||
**1. Interactive Mode** - We'll work through each section together, discussing and refining as we go
|
||||
**2. YOLO Mode** - I'll generate a complete draft based on our conversation so far, then we'll refine it together
|
||||
|
||||
Which approach works best for you?</ask>
|
||||
|
||||
<action>Store the user's preference for mode</action>
|
||||
<template-output>collaboration_mode</template-output>
|
||||
</step>
|
||||
|
||||
<step n="3" goal="Define game vision" if="collaboration_mode == 'interactive'">
|
||||
<action>Guide user to articulate their game vision across three levels of depth</action>
|
||||
<action>Help them craft a one-sentence core concept that captures the essence (reference successful games like "A roguelike deck-builder where you climb a mysterious spire" as examples)</action>
|
||||
<action>Develop an elevator pitch (2-3 sentences) that would compel a publisher or player - refine until it's concise but hooks attention</action>
|
||||
<action>Explore their aspirational vision statement: the experience they want to create and what makes it meaningful - ensure it's ambitious yet achievable</action>
|
||||
<action>Refine through conversation, challenging vague language and elevating compelling ideas</action>
|
||||
|
||||
<template-output>core_concept</template-output>
|
||||
<template-output>elevator_pitch</template-output>
|
||||
<template-output>vision_statement</template-output>
|
||||
</step>
|
||||
|
||||
<step n="4" goal="Identify target market" if="collaboration_mode == 'interactive'">
|
||||
<action>Guide user to define their primary target audience with specific demographics, gaming preferences, and behavioral characteristics</action>
|
||||
<action>Push for specificity beyond generic descriptions like "people who like fun games" - challenge vague answers</action>
|
||||
<action>Explore secondary audiences if applicable and how their needs might differ</action>
|
||||
<action>Investigate the market context: opportunity size, competitive landscape, similar successful games, and why now is the right time</action>
|
||||
<action>Help identify a realistic and reachable audience segment based on evidence or well-reasoned assumptions</action>
|
||||
|
||||
<template-output>primary_audience</template-output>
|
||||
<template-output>secondary_audience</template-output>
|
||||
<template-output>market_context</template-output>
|
||||
</step>
|
||||
|
||||
<step n="5" goal="Define game fundamentals" if="collaboration_mode == 'interactive'">
|
||||
<action>Help user identify 2-4 core gameplay pillars that fundamentally define their game - everything should support these pillars</action>
|
||||
<action>Provide examples from successful games for inspiration (Hollow Knight's "tight controls + challenging combat + rewarding exploration")</action>
|
||||
<action>Explore what the player actually DOES - core actions, key systems, and interaction models</action>
|
||||
<action>Define the emotional experience goals: what feelings are you designing for (tension/relief, mastery/growth, creativity/expression, discovery/surprise)</action>
|
||||
<action>Ensure pillars are specific and measurable, focusing on player actions rather than implementation details</action>
|
||||
<action>Connect mechanics directly to emotional experiences through guided discussion</action>
|
||||
|
||||
<template-output>core_gameplay_pillars</template-output>
|
||||
<template-output>primary_mechanics</template-output>
|
||||
<template-output>player_experience_goals</template-output>
|
||||
</step>
|
||||
|
||||
<step n="6" goal="Define scope and constraints" if="collaboration_mode == 'interactive'">
|
||||
<action>Help user establish realistic project constraints across all key dimensions</action>
|
||||
<action>Explore target platforms and prioritization (PC, console, mobile, web)</action>
|
||||
<action>Discuss development timeline: release targets, fixed deadlines, phased release strategies</action>
|
||||
<action>Investigate budget reality: funding source, asset creation costs, marketing, tools and software</action>
|
||||
<action>Assess team resources: size, roles, availability, skills gaps, outsourcing needs</action>
|
||||
<action>Define technical constraints: engine choice, performance targets, file size limits, accessibility requirements</action>
|
||||
<action>Push for realism about scope - identify potential blockers early and document resource assumptions</action>
|
||||
|
||||
<template-output>target_platforms</template-output>
|
||||
<template-output>development_timeline</template-output>
|
||||
<template-output>budget_considerations</template-output>
|
||||
<template-output>team_resources</template-output>
|
||||
<template-output>technical_constraints</template-output>
|
||||
</step>
|
||||
|
||||
<step n="7" goal="Establish reference framework" if="collaboration_mode == 'interactive'">
|
||||
<action>Guide user to identify 3-5 inspiration games and articulate what they're drawing from each (mechanics, feel, art style) and explicitly what they're NOT taking</action>
|
||||
<action>Conduct competitive analysis: identify direct and indirect competitors, analyze what they do well and poorly, and define how this game will differ</action>
|
||||
<action>Explore key differentiators and unique value proposition - what's the hook that makes players choose this game over alternatives</action>
|
||||
<action>Challenge "just better" thinking - push for genuine, specific differentiation that's actually valuable to players</action>
|
||||
<action>Validate that differentiators are concrete, achievable, and compelling</action>
|
||||
|
||||
<template-output>inspiration_games</template-output>
|
||||
<template-output>competitive_analysis</template-output>
|
||||
<template-output>key_differentiators</template-output>
|
||||
</step>
|
||||
|
||||
<step n="8" goal="Define content framework" if="collaboration_mode == 'interactive'">
|
||||
<action>Explore the game's world and setting: location, time period, world-building depth, narrative importance, and genre context</action>
|
||||
<action>Define narrative approach: story-driven/light/absent, linear/branching/emergent, delivery methods (cutscenes, dialogue, environmental), writing scope</action>
|
||||
<action>Estimate content volume realistically: playthrough length, level/stage count, replayability strategy, total asset volume</action>
|
||||
<action>Identify if a dedicated narrative workflow will be needed later based on story complexity</action>
|
||||
<action>Flag content-heavy areas that require detailed planning and resource allocation</action>
|
||||
|
||||
<template-output>world_setting</template-output>
|
||||
<template-output>narrative_approach</template-output>
|
||||
<template-output>content_volume</template-output>
|
||||
</step>
|
||||
|
||||
<step n="9" goal="Define art and audio direction" if="collaboration_mode == 'interactive'">
|
||||
<action>Explore visual style direction: art style preference, color palette and mood, reference games/images, 2D vs 3D, animation requirements</action>
|
||||
<action>Define audio style: music genre and mood, SFX approach, voice acting scope, audio's importance to gameplay</action>
|
||||
<action>Discuss production approach: in-house creation vs outsourcing, asset store usage, AI/generative tools, style complexity vs team capability</action>
|
||||
<action>Ensure art and audio vision aligns realistically with budget and team skills - identify potential production bottlenecks early</action>
|
||||
<action>Note if a comprehensive style guide will be needed for consistent production</action>
|
||||
|
||||
<template-output>visual_style</template-output>
|
||||
<template-output>audio_style</template-output>
|
||||
<template-output>production_approach</template-output>
|
||||
</step>
|
||||
|
||||
<step n="10" goal="Assess risks" if="collaboration_mode == 'interactive'">
|
||||
<action>Facilitate honest risk assessment across all dimensions - what could prevent completion, what could make it unfun, what assumptions might be wrong</action>
|
||||
<action>Identify technical challenges: unproven elements, performance concerns, platform-specific issues, tool dependencies</action>
|
||||
<action>Explore market risks: saturation, trend dependency, competition intensity, discoverability challenges</action>
|
||||
<action>For each major risk, develop actionable mitigation strategies - how to validate assumptions, backup plans, early prototyping opportunities</action>
|
||||
<action>Prioritize risks by impact and likelihood, focusing on proactive mitigation rather than passive worry</action>
|
||||
|
||||
<template-output>key_risks</template-output>
|
||||
<template-output>technical_challenges</template-output>
|
||||
<template-output>market_risks</template-output>
|
||||
<template-output>mitigation_strategies</template-output>
|
||||
</step>
|
||||
|
||||
<step n="11" goal="Define success criteria" if="collaboration_mode == 'interactive'">
|
||||
<action>Define the MVP (Minimum Playable Version) - what's the absolute minimum where the core loop is fun and complete, with essential content only</action>
|
||||
<action>Establish specific, measurable success metrics: player acquisition, retention rates, session length, completion rate, review scores, revenue targets, community engagement</action>
|
||||
<action>Set concrete launch goals: first-month sales/downloads, review score targets, streamer/press coverage, community size</action>
|
||||
<action>Push for specificity and measurability - challenge vague aspirations with "how will you measure that?"</action>
|
||||
<action>Clearly distinguish between MVP milestones and full release goals, ensuring all targets are realistic given resources</action>
|
||||
|
||||
<template-output>mvp_definition</template-output>
|
||||
<template-output>success_metrics</template-output>
|
||||
<template-output>launch_goals</template-output>
|
||||
</step>
|
||||
|
||||
<step n="12" goal="Identify immediate next steps" if="collaboration_mode == 'interactive'">
|
||||
<action>Identify immediate actions to take right after this brief: prototype core mechanics, create art style tests, validate technical feasibility, build vertical slice, playtest with target audience</action>
|
||||
<action>Determine research needs: market validation, technical proof of concept, player interest testing, competitive deep-dive</action>
|
||||
<action>Document open questions and uncertainties: unresolved design questions, technical unknowns, market validation needs, resource/budget questions</action>
|
||||
<action>Create actionable, specific next steps - prioritize by importance and dependency</action>
|
||||
<action>Identify blockers that must be resolved before moving forward</action>
|
||||
|
||||
<template-output>immediate_actions</template-output>
|
||||
<template-output>research_needs</template-output>
|
||||
<template-output>open_questions</template-output>
|
||||
</step>
|
||||
|
||||
<!-- YOLO Mode - Generate everything then refine -->
|
||||
<step n="3" goal="Generate complete brief draft" if="collaboration_mode == 'yolo'">
|
||||
<action>Based on initial context and any provided documents, generate a complete game brief covering all sections</action>
|
||||
<action>Make reasonable assumptions where information is missing</action>
|
||||
<action>Flag areas that need user validation with [NEEDS CONFIRMATION] tags</action>
|
||||
|
||||
<template-output>core_concept</template-output>
|
||||
<template-output>elevator_pitch</template-output>
|
||||
<template-output>vision_statement</template-output>
|
||||
<template-output>primary_audience</template-output>
|
||||
<template-output>secondary_audience</template-output>
|
||||
<template-output>market_context</template-output>
|
||||
<template-output>core_gameplay_pillars</template-output>
|
||||
<template-output>primary_mechanics</template-output>
|
||||
<template-output>player_experience_goals</template-output>
|
||||
<template-output>target_platforms</template-output>
|
||||
<template-output>development_timeline</template-output>
|
||||
<template-output>budget_considerations</template-output>
|
||||
<template-output>team_resources</template-output>
|
||||
<template-output>technical_constraints</template-output>
|
||||
<template-output>inspiration_games</template-output>
|
||||
<template-output>competitive_analysis</template-output>
|
||||
<template-output>key_differentiators</template-output>
|
||||
<template-output>world_setting</template-output>
|
||||
<template-output>narrative_approach</template-output>
|
||||
<template-output>content_volume</template-output>
|
||||
<template-output>visual_style</template-output>
|
||||
<template-output>audio_style</template-output>
|
||||
<template-output>production_approach</template-output>
|
||||
<template-output>key_risks</template-output>
|
||||
<template-output>technical_challenges</template-output>
|
||||
<template-output>market_risks</template-output>
|
||||
<template-output>mitigation_strategies</template-output>
|
||||
<template-output>mvp_definition</template-output>
|
||||
<template-output>success_metrics</template-output>
|
||||
<template-output>launch_goals</template-output>
|
||||
<template-output>immediate_actions</template-output>
|
||||
<template-output>research_needs</template-output>
|
||||
<template-output>open_questions</template-output>
|
||||
|
||||
<action>Present the complete draft to the user</action>
|
||||
<ask>Here's the complete game brief draft. What would you like to adjust or refine?</ask>
|
||||
</step>
|
||||
|
||||
<step n="4" goal="Refine brief sections" repeat="until-approved" if="collaboration_mode == 'yolo'">
|
||||
<ask>Which section would you like to refine?
|
||||
|
||||
1. Game Vision
|
||||
2. Target Market
|
||||
3. Game Fundamentals
|
||||
4. Scope and Constraints
|
||||
5. Reference Framework
|
||||
6. Content Framework
|
||||
7. Art and Audio Direction
|
||||
8. Risk Assessment
|
||||
9. Success Criteria
|
||||
10. Next Steps
|
||||
11. Save and continue</ask>
|
||||
|
||||
<action>Work with user to refine selected section</action>
|
||||
<action>Update relevant template outputs</action>
|
||||
</step>
|
||||
|
||||
<!-- Final steps for both modes -->
|
||||
<step n="13" goal="Create executive summary">
|
||||
<action>Synthesize all sections into a compelling executive summary</action>
|
||||
<action>Include:
|
||||
- Game concept in 1-2 sentences
|
||||
- Target audience and market
|
||||
- Core gameplay pillars
|
||||
- Key differentiators
|
||||
- Success vision</action>
|
||||
|
||||
<template-output>executive_summary</template-output>
|
||||
</step>
|
||||
|
||||
<step n="14" goal="Compile supporting materials">
|
||||
<action>If research documents were provided, create a summary of key findings</action>
|
||||
<action>Document any stakeholder input received during the process</action>
|
||||
<action>Compile list of reference games and resources</action>
|
||||
|
||||
<template-output>research_summary</template-output>
|
||||
<template-output>stakeholder_input</template-output>
|
||||
<template-output>references</template-output>
|
||||
</step>
|
||||
|
||||
<step n="15" goal="Final review and handoff">
|
||||
<action>Generate the complete game brief document</action>
|
||||
<action>Review all sections for completeness and consistency</action>
|
||||
<action>Flag any areas that need design attention with [DESIGN-TODO] tags</action>
|
||||
|
||||
<ask>The game brief is complete! Would you like to:
|
||||
|
||||
1. Review the entire document
|
||||
2. Make final adjustments
|
||||
3. Generate an executive summary version (3-page limit)
|
||||
4. Save and prepare for GDD creation
|
||||
|
||||
This brief will serve as the primary input for creating the Game Design Document (GDD).
|
||||
|
||||
**Recommended next steps:**
|
||||
|
||||
- Create prototype of core mechanic
|
||||
- Proceed to GDD workflow: `workflow gdd`
|
||||
- Validate assumptions with target players</ask>
|
||||
|
||||
<check if="user chooses option 3 (executive summary)">
|
||||
<action>Create condensed 3-page executive brief focusing on: core concept, target market, gameplay pillars, key differentiators, and success criteria</action>
|
||||
<action>Save as: {output_folder}/game-brief-executive-{{game_name}}-{{date}}.md</action>
|
||||
</check>
|
||||
|
||||
<template-output>final_brief</template-output>
|
||||
<template-output>executive_brief</template-output>
|
||||
</step>
|
||||
|
||||
<step n="16" goal="Update status and complete" tag="workflow-status">
|
||||
<check if="standalone_mode != true">
|
||||
<action>Load the FULL file: {output_folder}/bmgd-workflow-status.yaml</action>
|
||||
<action>Find workflow_status key "game-brief"</action>
|
||||
<critical>ONLY write the file path as the status value - no other text, notes, or metadata</critical>
|
||||
<action>Update workflow_status["game-brief"] = "{output_folder}/bmm-game-brief-{{game_name}}-{{date}}.md"</action>
|
||||
<action>Save file, preserving ALL comments and structure including STATUS DEFINITIONS</action>
|
||||
|
||||
<action>Find first non-completed workflow in workflow_status (next workflow to do)</action>
|
||||
<action>Determine next agent from path file based on next workflow</action>
|
||||
</check>
|
||||
|
||||
<output>**✅ Game Brief Complete, {user_name}!**
|
||||
|
||||
**Brief Document:**
|
||||
|
||||
- Game brief saved to {output_folder}/bmm-game-brief-{{game_name}}-{{date}}.md
|
||||
|
||||
{{#if standalone_mode != true}}
|
||||
**Status Updated:**
|
||||
|
||||
- Progress tracking updated: game-brief marked complete
|
||||
- Next workflow: {{next_workflow}}
|
||||
{{else}}
|
||||
**Note:** Running in standalone mode (no progress tracking)
|
||||
{{/if}}
|
||||
|
||||
**Next Steps:**
|
||||
|
||||
{{#if standalone_mode != true}}
|
||||
|
||||
- **Next workflow:** {{next_workflow}} ({{next_agent}} agent)
|
||||
- **Optional:** Consider creating a prototype of core mechanic or validating assumptions with target players before proceeding
|
||||
|
||||
Check status anytime with: `workflow-status`
|
||||
{{else}}
|
||||
Since no workflow is in progress:
|
||||
|
||||
- Refer to the BMM workflow guide if unsure what to do next
|
||||
- Or run `workflow-init` to create a workflow path and get guided next steps
|
||||
{{/if}}
|
||||
</output>
|
||||
</step>
|
||||
|
||||
</workflow>
|
||||
|
|
@ -1,224 +0,0 @@
|
|||
---
|
||||
name: 'step-01-init'
|
||||
description: 'Initialize the Game Brief workflow by detecting continuation state and setting up the document'
|
||||
|
||||
# Path Definitions
|
||||
workflow_path: '{project-root}/_bmad/bmgd/workflows/1-preproduction/game-brief'
|
||||
|
||||
# File References
|
||||
thisStepFile: './step-01-init.md'
|
||||
nextStepFile: './step-02-vision.md'
|
||||
continueStepFile: './step-01b-continue.md'
|
||||
workflowFile: '{workflow_path}/workflow.md'
|
||||
outputFile: '{output_folder}/game-brief.md'
|
||||
|
||||
# Template References
|
||||
briefTemplate: '{workflow_path}/templates/game-brief-template.md'
|
||||
---
|
||||
|
||||
# Step 1: Workflow Initialization
|
||||
|
||||
**Progress: Step 1 of 8** - Next: Game Vision
|
||||
|
||||
## STEP GOAL:
|
||||
|
||||
Initialize the Game Brief workflow by detecting continuation state, discovering any input documents (brainstorming, research), and setting up the document structure.
|
||||
|
||||
## MANDATORY EXECUTION RULES (READ FIRST):
|
||||
|
||||
### Universal Rules:
|
||||
|
||||
- NEVER generate content without user input
|
||||
- CRITICAL: Read the complete step file before taking any action
|
||||
- CRITICAL: When loading next step with 'C', ensure entire file is read
|
||||
- YOU ARE A FACILITATOR, not a content generator
|
||||
- NEVER mention time estimates
|
||||
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
||||
|
||||
### Role Reinforcement:
|
||||
|
||||
- You are a veteran game designer facilitator collaborating with a creative peer
|
||||
- If you already have been given a name, communication_style and persona, continue to use those while playing this new role
|
||||
- We engage in collaborative dialogue, not command-response
|
||||
- You bring structured game design thinking and facilitation skills
|
||||
|
||||
### Step-Specific Rules:
|
||||
|
||||
- Focus only on initialization and setup - no content generation yet
|
||||
- FORBIDDEN to look ahead to future steps or assume knowledge from them
|
||||
- Approach: Systematic setup with clear reporting to user
|
||||
- Detect existing workflow state and handle continuation properly
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- Show your analysis of current state before taking any action
|
||||
- Initialize document structure and update frontmatter appropriately
|
||||
- Set up frontmatter `stepsCompleted: [1]` before loading next step
|
||||
- FORBIDDEN to load next step until user selects 'C' (Continue)
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: Variables from workflow.md are available in memory
|
||||
- Focus: Workflow initialization and document setup only
|
||||
- Limits: Don't assume knowledge from other steps or create content yet
|
||||
- Dependencies: Configuration loaded from workflow.md initialization
|
||||
|
||||
## Sequence of Instructions (Do not deviate, skip, or optimize)
|
||||
|
||||
### 1. Check for Existing Workflow State
|
||||
|
||||
First, check if the output document already exists:
|
||||
|
||||
**Workflow State Detection:**
|
||||
|
||||
- Look for file at `{outputFile}`
|
||||
- If exists, read the complete file including frontmatter
|
||||
- If not exists, this is a fresh workflow
|
||||
|
||||
### 2. Handle Continuation (If Document Exists)
|
||||
|
||||
If the document exists and has frontmatter with `stepsCompleted`:
|
||||
|
||||
**Continuation Protocol:**
|
||||
|
||||
- **STOP immediately** and load `{continueStepFile}`
|
||||
- Do not proceed with any initialization tasks
|
||||
- Let step-01b handle all continuation logic
|
||||
- This is an auto-proceed situation - no user choice needed
|
||||
|
||||
### 3. Fresh Workflow Setup (If No Document)
|
||||
|
||||
If no document exists or no `stepsCompleted` in frontmatter:
|
||||
|
||||
#### A. Input Document Discovery
|
||||
|
||||
Discover and load context documents using smart discovery.
|
||||
|
||||
**IMPORTANT: Track document counts as you discover files.**
|
||||
|
||||
Initialize counters:
|
||||
|
||||
```
|
||||
brainstormingCount = 0
|
||||
researchCount = 0
|
||||
notesCount = 0
|
||||
```
|
||||
|
||||
**Brainstorming Documents:**
|
||||
|
||||
1. Check: `{output_folder}/*brainstorm*.md`
|
||||
2. Check: `{output_folder}/analysis/*brainstorm*.md`
|
||||
3. Load completely and extract key ideas
|
||||
4. **Update brainstormingCount with number of files found**
|
||||
|
||||
**Research Documents:**
|
||||
|
||||
1. Check: `{output_folder}/*research*.md`
|
||||
2. Check: `{output_folder}/analysis/*research*.md`
|
||||
3. Load useful research files completely
|
||||
4. **Update researchCount with number of files found**
|
||||
|
||||
**Design Notes:**
|
||||
|
||||
1. Check: `{output_folder}/*notes*.md` or `{output_folder}/*design*.md`
|
||||
2. Load any relevant design notes
|
||||
3. **Update notesCount with number of files found**
|
||||
|
||||
**Loading Rules:**
|
||||
|
||||
- Load ALL discovered files completely (no offset/limit)
|
||||
- Track all successfully loaded files in frontmatter `inputDocuments` array
|
||||
|
||||
#### B. Create Initial Document
|
||||
|
||||
**Document Setup:**
|
||||
|
||||
- Copy the template from `{briefTemplate}` to `{outputFile}`
|
||||
- Initialize frontmatter with proper structure:
|
||||
|
||||
```yaml
|
||||
---
|
||||
stepsCompleted: []
|
||||
inputDocuments: []
|
||||
documentCounts:
|
||||
brainstorming: { { brainstormingCount } }
|
||||
research: { { researchCount } }
|
||||
notes: { { notesCount } }
|
||||
workflowType: 'game-brief'
|
||||
lastStep: 0
|
||||
project_name: '{{project_name}}'
|
||||
user_name: '{{user_name}}'
|
||||
date: '{{date}}'
|
||||
game_name: ''
|
||||
---
|
||||
```
|
||||
|
||||
#### C. Present Initialization Results
|
||||
|
||||
**Setup Report to User:**
|
||||
|
||||
"Welcome {{user_name}}! I've set up your Game Brief workspace.
|
||||
|
||||
**Document Setup:**
|
||||
|
||||
- Created: `{outputFile}` from template
|
||||
- Initialized frontmatter with workflow state
|
||||
|
||||
**Input Documents Discovered:**
|
||||
|
||||
- Brainstorming: {{brainstormingCount}} files {if brainstormingCount > 0}loaded{else}(none found){/if}
|
||||
- Research: {{researchCount}} files {if researchCount > 0}loaded{else}(none found){/if}
|
||||
- Design notes: {{notesCount}} files {if notesCount > 0}loaded{else}(none found){/if}
|
||||
|
||||
{if any_documents_found}
|
||||
I'll use these documents to give us a head start on your game brief.
|
||||
{else}
|
||||
We'll start fresh and build your game vision together through conversation.
|
||||
{/if}
|
||||
|
||||
Do you have any other documents you'd like me to include, or shall we continue?"
|
||||
|
||||
### 4. Present MENU OPTIONS
|
||||
|
||||
Display menu after setup report:
|
||||
|
||||
"[C] Continue - Save this and move to Game Vision (Step 2 of 8)"
|
||||
|
||||
#### Menu Handling Logic:
|
||||
|
||||
- IF C: Update frontmatter with `stepsCompleted: [1]`, then load, read entire file, then execute {nextStepFile}
|
||||
- IF user provides additional files: Load them, update inputDocuments and documentCounts, redisplay report
|
||||
- IF user asks questions: Answer and redisplay menu
|
||||
|
||||
#### EXECUTION RULES:
|
||||
|
||||
- ALWAYS halt and wait for user input after presenting menu
|
||||
- ONLY proceed to next step when user selects 'C'
|
||||
|
||||
## CRITICAL STEP COMPLETION NOTE
|
||||
|
||||
ONLY WHEN [C continue option] is selected and [frontmatter properly updated with stepsCompleted: [1] and documentCounts], will you then load and read fully `{nextStepFile}` to execute and begin game vision discovery.
|
||||
|
||||
---
|
||||
|
||||
## SYSTEM SUCCESS/FAILURE METRICS
|
||||
|
||||
### SUCCESS:
|
||||
|
||||
- Existing workflow detected and properly handed off to step-01b
|
||||
- Fresh workflow initialized with template and proper frontmatter
|
||||
- Input documents discovered and loaded
|
||||
- All discovered files tracked in frontmatter `inputDocuments`
|
||||
- **Document counts stored in frontmatter `documentCounts`**
|
||||
- Menu presented and user input handled correctly
|
||||
- Frontmatter updated with `stepsCompleted: [1]` before proceeding
|
||||
|
||||
### SYSTEM FAILURE:
|
||||
|
||||
- Proceeding with fresh initialization when existing workflow exists
|
||||
- Not updating frontmatter with discovered input documents
|
||||
- **Not storing document counts in frontmatter**
|
||||
- Creating document without proper template structure
|
||||
- Proceeding without user selecting 'C' (Continue)
|
||||
|
||||
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.
|
||||
|
|
@ -1,152 +0,0 @@
|
|||
---
|
||||
name: 'step-01b-continue'
|
||||
description: 'Resume an interrupted Game Brief workflow from the last completed step'
|
||||
|
||||
# Path Definitions
|
||||
workflow_path: '{project-root}/_bmad/bmgd/workflows/1-preproduction/game-brief'
|
||||
|
||||
# File References
|
||||
thisStepFile: './step-01b-continue.md'
|
||||
workflowFile: '{workflow_path}/workflow.md'
|
||||
outputFile: '{output_folder}/game-brief.md'
|
||||
---
|
||||
|
||||
# Step 1B: Workflow Continuation
|
||||
|
||||
## STEP GOAL:
|
||||
|
||||
Resume the Game Brief workflow from where it was left off, ensuring smooth continuation with full context restoration.
|
||||
|
||||
## MANDATORY EXECUTION RULES (READ FIRST):
|
||||
|
||||
### Universal Rules:
|
||||
|
||||
- NEVER generate content without user input
|
||||
- CRITICAL: Read the complete step file before taking any action
|
||||
- CRITICAL: When loading next step with 'C', ensure entire file is read
|
||||
- YOU ARE A FACILITATOR, not a content generator
|
||||
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
||||
|
||||
### Role Reinforcement:
|
||||
|
||||
- You are a veteran game designer facilitator collaborating with a creative peer
|
||||
- We engage in collaborative dialogue, not command-response
|
||||
- Resume workflow from exact point where it was interrupted
|
||||
|
||||
### Step-Specific Rules:
|
||||
|
||||
- FOCUS on understanding where we left off and continuing appropriately
|
||||
- FORBIDDEN to modify content completed in previous steps
|
||||
- Only reload documents that were already tracked in `inputDocuments`
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- Show your analysis of current state before taking action
|
||||
- Keep existing frontmatter `stepsCompleted` values
|
||||
- Only load documents that were already tracked in `inputDocuments`
|
||||
- FORBIDDEN to discover new input documents during continuation
|
||||
|
||||
## Sequence of Instructions (Do not deviate, skip, or optimize)
|
||||
|
||||
### 1. Analyze Current State
|
||||
|
||||
**State Assessment:**
|
||||
Review the frontmatter to understand:
|
||||
|
||||
- `stepsCompleted`: Which steps are already done
|
||||
- `lastStep`: The most recently completed step number
|
||||
- `inputDocuments`: What context was already loaded
|
||||
- `documentCounts`: brainstorming, research, notes counts
|
||||
- `game_name`: The game name (if set)
|
||||
- All other frontmatter variables
|
||||
|
||||
### 2. Restore Context Documents
|
||||
|
||||
**Context Reloading:**
|
||||
|
||||
- For each document in `inputDocuments`, load the complete file
|
||||
- This ensures you have full context for continuation
|
||||
- Don't discover new documents - only reload what was previously processed
|
||||
|
||||
### 3. Present Current Progress
|
||||
|
||||
**Progress Report to User:**
|
||||
"Welcome back {{user_name}}! I'm resuming our Game Brief collaboration for {{game_name or project_name}}.
|
||||
|
||||
**Current Progress:**
|
||||
|
||||
- Steps completed: {stepsCompleted}
|
||||
- Last worked on: Step {lastStep}
|
||||
- Context documents available: {len(inputDocuments)} files
|
||||
|
||||
**Document Status:**
|
||||
|
||||
- Current Game Brief is ready with all completed sections
|
||||
- Ready to continue from where we left off
|
||||
|
||||
Does this look right, or do you want to make any adjustments before we proceed?"
|
||||
|
||||
### 4. Determine Continuation Path
|
||||
|
||||
**Next Step Logic:**
|
||||
Based on `lastStep` value, determine which step to load next:
|
||||
|
||||
- If `lastStep = 1` -> Load `./step-02-vision.md`
|
||||
- If `lastStep = 2` -> Load `./step-03-market.md`
|
||||
- If `lastStep = 3` -> Load `./step-04-fundamentals.md`
|
||||
- If `lastStep = 4` -> Load `./step-05-scope.md`
|
||||
- If `lastStep = 5` -> Load `./step-06-references.md`
|
||||
- If `lastStep = 6` -> Load `./step-07-content.md`
|
||||
- If `lastStep = 7` -> Load `./step-08-complete.md`
|
||||
- If `lastStep = 8` -> Workflow already complete
|
||||
|
||||
### 5. Handle Workflow Completion
|
||||
|
||||
**If workflow already complete (`lastStep = 8`):**
|
||||
"Great news! It looks like we've already completed the Game Brief workflow for {{game_name}}.
|
||||
|
||||
The final document is ready at `{outputFile}` with all sections completed.
|
||||
|
||||
Would you like me to:
|
||||
|
||||
- Review the completed brief with you
|
||||
- Suggest next workflow steps (like GDD creation)
|
||||
- Start a new brief revision
|
||||
|
||||
What would be most helpful?"
|
||||
|
||||
### 6. Present MENU OPTIONS
|
||||
|
||||
**If workflow not complete:**
|
||||
Display: "Ready to continue with Step {nextStepNumber}?
|
||||
|
||||
**Select an Option:** [C] Continue to next step"
|
||||
|
||||
#### Menu Handling Logic:
|
||||
|
||||
- IF C: Load, read entire file, then execute the appropriate next step file based on `lastStep`
|
||||
- IF Any other comments or queries: respond and redisplay menu
|
||||
|
||||
## CRITICAL STEP COMPLETION NOTE
|
||||
|
||||
ONLY WHEN [C continue option] is selected and [current state confirmed], will you then load and read fully the appropriate next step file to resume the workflow.
|
||||
|
||||
---
|
||||
|
||||
## SYSTEM SUCCESS/FAILURE METRICS
|
||||
|
||||
### SUCCESS:
|
||||
|
||||
- All previous input documents successfully reloaded
|
||||
- Current workflow state accurately analyzed and presented
|
||||
- User confirms understanding of progress before continuation
|
||||
- Correct next step identified and prepared for loading
|
||||
|
||||
### SYSTEM FAILURE:
|
||||
|
||||
- Discovering new input documents instead of reloading existing ones
|
||||
- Modifying content from already completed steps
|
||||
- Loading wrong next step based on `lastStep` value
|
||||
- Proceeding without user confirmation of current state
|
||||
|
||||
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.
|
||||
|
|
@ -1,219 +0,0 @@
|
|||
---
|
||||
name: 'step-02-vision'
|
||||
description: 'Define the core game vision including name, concept, pitch, and vision statement'
|
||||
|
||||
# Path Definitions
|
||||
workflow_path: '{project-root}/_bmad/bmgd/workflows/1-preproduction/game-brief'
|
||||
|
||||
# File References
|
||||
thisStepFile: './step-02-vision.md'
|
||||
nextStepFile: './step-03-market.md'
|
||||
workflowFile: '{workflow_path}/workflow.md'
|
||||
outputFile: '{output_folder}/game-brief.md'
|
||||
|
||||
# Task References
|
||||
advancedElicitationTask: '{project-root}/_bmad/core/workflows/advanced-elicitation/workflow.xml'
|
||||
partyModeWorkflow: '{project-root}/_bmad/core/workflows/party-mode/workflow.md'
|
||||
---
|
||||
|
||||
# Step 2: Game Vision
|
||||
|
||||
**Progress: Step 2 of 8** - Next: Target Market
|
||||
|
||||
## STEP GOAL:
|
||||
|
||||
Capture the core game vision including the working title, one-sentence concept, elevator pitch, and aspirational vision statement that will guide all design decisions.
|
||||
|
||||
## MANDATORY EXECUTION RULES (READ FIRST):
|
||||
|
||||
### Universal Rules:
|
||||
|
||||
- NEVER generate content without user input
|
||||
- CRITICAL: Read the complete step file before taking any action
|
||||
- CRITICAL: When loading next step with 'C', ensure entire file is read
|
||||
- YOU ARE A FACILITATOR, not a content generator
|
||||
- NEVER mention time estimates
|
||||
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
||||
|
||||
### Role Reinforcement:
|
||||
|
||||
- You are a veteran game designer facilitator collaborating with a creative peer
|
||||
- Vision is the foundation - get it right before moving forward
|
||||
- Challenge vague language and elevate compelling ideas
|
||||
|
||||
### Step-Specific Rules:
|
||||
|
||||
- Focus on crystallizing the game's core identity
|
||||
- FORBIDDEN to generate vision without real user input
|
||||
- Push for specificity and clarity
|
||||
- Reference successful games as examples of good pitches
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- Show your analysis before taking any action
|
||||
- Present A/P/C menu after generating content
|
||||
- ONLY save when user chooses C (Continue)
|
||||
- Update frontmatter `stepsCompleted: [1, 2]` before loading next step
|
||||
- FORBIDDEN to load next step until C is selected
|
||||
|
||||
## COLLABORATION MENUS (A/P/C):
|
||||
|
||||
- **A (Advanced Elicitation)**: Dig deeper into the vision
|
||||
- **P (Party Mode)**: Get multiple perspectives on the vision
|
||||
- **C (Continue)**: Save the content to the document and proceed to next step
|
||||
|
||||
## Sequence of Instructions (Do not deviate, skip, or optimize)
|
||||
|
||||
### 1. Game Name Discovery
|
||||
|
||||
**Start with the basics:**
|
||||
|
||||
"Let's start with your game's identity.
|
||||
|
||||
**What's the working title for your game?**
|
||||
|
||||
(Don't worry if it's not final - working titles help us talk about the game and can always change later.)"
|
||||
|
||||
Store in frontmatter: `game_name: '{user_provided_name}'`
|
||||
|
||||
### 2. Context Check
|
||||
|
||||
**If input documents were loaded:**
|
||||
|
||||
"I've reviewed your {brainstorming/research} documents and noticed some interesting ideas:
|
||||
|
||||
{summarize_key_themes_from_documents}
|
||||
|
||||
Let's use these as a starting point for crystallizing your vision."
|
||||
|
||||
### 3. Core Concept Discovery
|
||||
|
||||
**Guide user through concept definition:**
|
||||
|
||||
"Now let's capture the essence of {{game_name}} in a single sentence.
|
||||
|
||||
**Examples of great one-sentence concepts:**
|
||||
|
||||
- 'A roguelike deck-builder where you climb a mysterious spire' (Slay the Spire)
|
||||
- 'A precision platformer about climbing a mountain and overcoming anxiety' (Celeste)
|
||||
- 'A cozy farming sim where you rebuild your grandfather's farm and become part of a small town' (Stardew Valley)
|
||||
|
||||
**What is {{game_name}}?** Give me one sentence that captures the core experience."
|
||||
|
||||
### 4. Elevator Pitch Discovery
|
||||
|
||||
**Build on the concept:**
|
||||
|
||||
"Now let's expand that into an elevator pitch - 2-3 sentences that would compel a player or publisher to want to know more.
|
||||
|
||||
**A great elevator pitch answers:**
|
||||
|
||||
- What is it? (genre, style)
|
||||
- What do you do? (core action)
|
||||
- What makes it special? (hook)
|
||||
|
||||
**Refine this until it hooks attention.** What's your elevator pitch for {{game_name}}?"
|
||||
|
||||
### 5. Vision Statement Discovery
|
||||
|
||||
**Explore the aspirational vision:**
|
||||
|
||||
"Finally, let's capture your aspirational vision - the experience you want to create and what makes it meaningful.
|
||||
|
||||
**Questions to consider:**
|
||||
|
||||
- What feeling do you want players to have when they put down the controller?
|
||||
- What would make this game matter to someone?
|
||||
- What's your personal motivation for making this?
|
||||
|
||||
**This is your North Star** - ambitious yet achievable. What's your vision for {{game_name}}?"
|
||||
|
||||
### 6. Generate Vision Content
|
||||
|
||||
Based on the conversation, prepare the content:
|
||||
|
||||
```markdown
|
||||
## Game Vision
|
||||
|
||||
### Core Concept
|
||||
|
||||
{{core_concept}}
|
||||
|
||||
### Elevator Pitch
|
||||
|
||||
{{elevator_pitch}}
|
||||
|
||||
### Vision Statement
|
||||
|
||||
{{vision_statement}}
|
||||
```
|
||||
|
||||
### 7. Present Content and Menu
|
||||
|
||||
Show the generated content to the user and present:
|
||||
|
||||
"I've drafted the Game Vision section based on our conversation.
|
||||
|
||||
**Here's what I'll add to the document:**
|
||||
|
||||
[Show the complete markdown content from step 6]
|
||||
|
||||
**Validation Check:**
|
||||
|
||||
- Does the core concept capture the essence?
|
||||
- Does the pitch hook attention?
|
||||
- Does the vision inspire?
|
||||
|
||||
**Select an Option:**
|
||||
[A] Advanced Elicitation - Refine and strengthen the vision
|
||||
[P] Party Mode - Get other perspectives on the vision
|
||||
[C] Continue - Save this and move to Target Market (Step 3 of 8)"
|
||||
|
||||
### 8. Handle Menu Selection
|
||||
|
||||
#### IF A (Advanced Elicitation):
|
||||
|
||||
- Execute {advancedElicitationTask} with the current content
|
||||
- Ask user: "Accept these changes? (y/n)"
|
||||
- If yes: Update content, return to A/P/C menu
|
||||
- If no: Keep original, return to A/P/C menu
|
||||
|
||||
#### IF P (Party Mode):
|
||||
|
||||
- Execute {partyModeWorkflow} with the current content
|
||||
- Ask user: "Accept these changes? (y/n)"
|
||||
- If yes: Update content, return to A/P/C menu
|
||||
- If no: Keep original, return to A/P/C menu
|
||||
|
||||
#### IF C (Continue):
|
||||
|
||||
- Append the final content to `{outputFile}`
|
||||
- Update frontmatter: `stepsCompleted: [1, 2]`, `game_name: '{game_name}'`
|
||||
- Load `{nextStepFile}`
|
||||
|
||||
## CRITICAL STEP COMPLETION NOTE
|
||||
|
||||
ONLY WHEN [C continue option] is selected and [vision content saved with frontmatter updated including game_name], will you then load and read fully `{nextStepFile}`.
|
||||
|
||||
---
|
||||
|
||||
## SYSTEM SUCCESS/FAILURE METRICS
|
||||
|
||||
### SUCCESS:
|
||||
|
||||
- Game name captured and stored in frontmatter
|
||||
- Core concept is clear and concise (one sentence)
|
||||
- Elevator pitch is compelling (2-3 sentences)
|
||||
- Vision statement is aspirational yet achievable
|
||||
- A/P/C menu presented and handled correctly
|
||||
- Frontmatter updated with stepsCompleted: [1, 2]
|
||||
|
||||
### SYSTEM FAILURE:
|
||||
|
||||
- Generating vision without user input
|
||||
- Core concept is vague or generic
|
||||
- Elevator pitch doesn't hook attention
|
||||
- Not presenting A/P/C menu after content generation
|
||||
- Proceeding without user selecting 'C'
|
||||
|
||||
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue