--- title: "How to Run Automate with TEA" description: Expand test automation coverage after implementation using TEA's automate workflow --- # How to Run Automate with TEA Use TEA's `automate` workflow to generate comprehensive tests for existing features. Unlike `*atdd`, these tests pass immediately because the feature already exists. ## When to Use This - Feature already exists and works - Want to add test coverage to existing code - Need tests that pass immediately - Expanding existing test suite - Adding tests to legacy code **Don't use this if:** - Feature doesn't exist yet (use `atdd` instead) - Want failing tests to guide development (use `atdd` for TDD) ## Prerequisites - BMad Method installed - TEA agent available - Test framework setup complete (run `framework` if needed) - Feature implemented and working **Note:** This guide uses Playwright examples. If using Cypress, commands and syntax will differ. ## Steps ### 1. Load TEA Agent Start a fresh chat and load TEA: ``` tea ``` ### 2. Run the Automate Workflow ``` automate ``` ### 3. Provide Context TEA will ask for context about what you're testing. #### Option A: BMad-Integrated Mode (Recommended) If you have BMad artifacts (stories, test designs, PRDs): **What are you testing?** ``` I'm testing the user profile feature we just implemented. Story: story-profile-management.md Test Design: test-design-epic-1.md ``` **Reference documents:** - Story file with acceptance criteria - Test design document (if available) - PRD sections relevant to this feature - Tech spec (if available) **Existing tests:** ``` We have basic tests in tests/e2e/profile-view.spec.ts Avoid duplicating that coverage ``` TEA will analyze your artifacts and generate comprehensive tests that: - Cover acceptance criteria from the story - Follow priorities from test design (P0 → P1 → P2) - Avoid duplicating existing tests - Include edge cases and error scenarios #### Option B: Standalone Mode If you're using TEA Solo or don't have BMad artifacts: **What are you testing?** ``` TodoMVC React application at https://todomvc.com/examples/react/dist/ Features: Create todos, mark as complete, filter by status, delete todos ``` **Specific scenarios to cover:** ``` - Creating todos (happy path) - Marking todos as complete/incomplete - Filtering (All, Active, Completed) - Deleting todos - Edge cases (empty input, long text) ``` TEA will analyze the application and generate tests based on your description. ### 4. Specify Test Levels TEA will ask which test levels to generate: **Options:** - **E2E tests** - Full browser-based user workflows - **API tests** - Backend endpoint testing (faster, more reliable) - **Component tests** - UI component testing in isolation (framework-dependent) - **Mix** - Combination of levels (recommended) **Example response:** ``` Generate: - API tests for all CRUD operations - E2E tests for critical user workflows (P0) - Focus on P0 and P1 scenarios - Skip P3 (low priority edge cases) ``` ### 5. Review Generated Tests TEA generates a comprehensive test suite with multiple test levels. #### API Tests (`tests/api/profile.spec.ts`): **Vanilla Playwright:** ```typescript import { test, expect } from '@playwright/test'; test.describe('Profile API', () => { let authToken: string; test.beforeAll(async ({ request }) => { // Manual auth token fetch const response = await request.post('/api/auth/login', { data: { email: 'test@example.com', password: 'password123' } }); const { token } = await response.json(); authToken = token; }); test('should fetch user profile', async ({ request }) => { const response = await request.get('/api/profile', { headers: { Authorization: `Bearer ${authToken}` } }); expect(response.ok()).toBeTruthy(); const profile = await response.json(); expect(profile).toMatchObject({ id: expect.any(String), name: expect.any(String), email: expect.any(String) }); }); test('should update profile successfully', async ({ request }) => { const response = await request.patch('/api/profile', { headers: { Authorization: `Bearer ${authToken}` }, data: { name: 'Updated Name', bio: 'Test bio' } }); expect(response.ok()).toBeTruthy(); const updated = await response.json(); expect(updated.name).toBe('Updated Name'); expect(updated.bio).toBe('Test bio'); }); test('should validate email format', async ({ request }) => { const response = await request.patch('/api/profile', { headers: { Authorization: `Bearer ${authToken}` }, data: { email: 'invalid-email' } }); expect(response.status()).toBe(400); const error = await response.json(); expect(error.message).toContain('Invalid email'); }); test('should require authentication', async ({ request }) => { const response = await request.get('/api/profile'); expect(response.status()).toBe(401); }); }); ``` **With Playwright Utils:** ```typescript import { test as base, expect } from '@playwright/test'; import { test as apiRequestFixture } from '@seontechnologies/playwright-utils/api-request/fixtures'; import { createAuthFixtures } from '@seontechnologies/playwright-utils/auth-session'; import { mergeTests } from '@playwright/test'; import { z } from 'zod'; const ProfileSchema = z.object({ id: z.string(), name: z.string(), email: z.string().email() }); // Merge API and auth fixtures const authFixtureTest = base.extend(createAuthFixtures()); export const testWithAuth = mergeTests(apiRequestFixture, authFixtureTest); testWithAuth.describe('Profile API', () => { testWithAuth('should fetch user profile', async ({ apiRequest, authToken }) => { const { status, body } = await apiRequest({ method: 'GET', path: '/api/profile', headers: { Authorization: `Bearer ${authToken}` } }).validateSchema(ProfileSchema); // Chained validation expect(status).toBe(200); // Schema already validated, type-safe access expect(body.name).toBeDefined(); }); testWithAuth('should update profile successfully', async ({ apiRequest, authToken }) => { const { status, body } = await apiRequest({ method: 'PATCH', path: '/api/profile', body: { name: 'Updated Name', bio: 'Test bio' }, headers: { Authorization: `Bearer ${authToken}` } }).validateSchema(ProfileSchema); // Chained validation expect(status).toBe(200); expect(body.name).toBe('Updated Name'); }); testWithAuth('should validate email format', async ({ apiRequest, authToken }) => { const { status, body } = await apiRequest({ method: 'PATCH', path: '/api/profile', body: { email: 'invalid-email' }, headers: { Authorization: `Bearer ${authToken}` } }); expect(status).toBe(400); expect(body.message).toContain('Invalid email'); }); }); ``` **Key Differences:** - `authToken` fixture (persisted, reused across tests) - `apiRequest` returns `{ status, body }` (cleaner) - Schema validation with Zod (type-safe) - Automatic retry for 5xx errors - Less boilerplate (no manual `await response.json()` everywhere) #### E2E Tests (`tests/e2e/profile.spec.ts`): ```typescript import { test, expect } from '@playwright/test'; test('should edit profile', async ({ page }) => { // Login await page.goto('/login'); await page.getByLabel('Email').fill('test@example.com'); await page.getByLabel('Password').fill('password123'); await page.getByRole('button', { name: 'Sign in' }).click(); // Edit profile await page.goto('/profile'); await page.getByRole('button', { name: 'Edit Profile' }).click(); await page.getByLabel('Name').fill('New Name'); await page.getByRole('button', { name: 'Save' }).click(); // Verify success await expect(page.getByText('Profile updated')).toBeVisible(); }); ``` TEA generates additional tests for validation, edge cases, etc. based on priorities. #### Fixtures (`tests/support/fixtures/profile.ts`): **Vanilla Playwright:** ```typescript import { test as base, Page } from '@playwright/test'; type ProfileFixtures = { authenticatedPage: Page; testProfile: { name: string; email: string; bio: string; }; }; export const test = base.extend({ authenticatedPage: async ({ page }, use) => { // Manual login flow await page.goto('/login'); await page.getByLabel('Email').fill('test@example.com'); await page.getByLabel('Password').fill('password123'); await page.getByRole('button', { name: 'Sign in' }).click(); await page.waitForURL(/\/dashboard/); await use(page); }, testProfile: async ({ request }, use) => { // Static test data const profile = { name: 'Test User', email: 'test@example.com', bio: 'Test bio' }; await use(profile); } }); ``` **With Playwright Utils:** ```typescript import { test as base } from '@playwright/test'; import { createAuthFixtures } from '@seontechnologies/playwright-utils/auth-session'; import { mergeTests } from '@playwright/test'; import { faker } from '@faker-js/faker'; type ProfileFixtures = { testProfile: { name: string; email: string; bio: string; }; }; // Merge auth fixtures with custom fixtures const authTest = base.extend(createAuthFixtures()); const profileTest = base.extend({ testProfile: async ({}, use) => { // Dynamic test data with faker const profile = { name: faker.person.fullName(), email: faker.internet.email(), bio: faker.person.bio() }; await use(profile); } }); export const test = mergeTests(authTest, profileTest); export { expect } from '@playwright/test'; ``` **Usage:** ```typescript import { test, expect } from '../support/fixtures/profile'; test('should update profile', async ({ page, authToken, testProfile }) => { // authToken from auth-session (automatic, persisted) // testProfile from custom fixture (dynamic data) await page.goto('/profile'); // Test with dynamic, unique data }); ``` **Key Benefits:** - `authToken` fixture (persisted token, no manual login) - Dynamic test data with faker (no conflicts) - Fixture composition with mergeTests - Reusable across test files ### 6. Review Additional Artifacts TEA also generates: #### Updated README (`tests/README.md`): ```markdown # Test Suite ## Running Tests ### All Tests npm test ### Specific Levels npm run test:api # API tests only npm run test:e2e # E2E tests only npm run test:smoke # Smoke tests (@smoke tag) ### Single File npx playwright test tests/api/profile.spec.ts ## Test Structure tests/ ├── api/ # API tests (fast, reliable) ├── e2e/ # E2E tests (full workflows) ├── fixtures/ # Shared test utilities └── README.md ## Writing Tests Follow the patterns in existing tests: - Use fixtures for authentication - Network-first patterns (no hard waits) - Explicit assertions - Self-cleaning tests ``` #### Definition of Done Summary: ```markdown ## Test Quality Checklist ✅ All tests pass on first run ✅ No hard waits (waitForTimeout) ✅ No conditionals for flow control ✅ Assertions are explicit ✅ Tests clean up after themselves ✅ Tests can run in parallel ✅ Execution time < 1.5 minutes per test ✅ Test files < 300 lines ``` ### 7. Run the Tests All tests should pass immediately since the feature exists: **For Playwright:** ```bash npx playwright test ``` **For Cypress:** ```bash npx cypress run ``` Expected output: ``` Running 15 tests using 4 workers ✓ tests/api/profile.spec.ts (4 tests) - 2.1s ✓ tests/e2e/profile-workflow.spec.ts (2 tests) - 5.3s 15 passed (7.4s) ``` **All green!** Tests pass because feature already exists. ### 8. Review Test Coverage Check which scenarios are covered: ```bash # View test report npx playwright show-report # Check coverage (if configured) npm run test:coverage ``` Compare against: - Acceptance criteria from story - Test priorities from test design - Edge cases and error scenarios ## What You Get ### Comprehensive Test Suite - **API tests** - Fast, reliable backend testing - **E2E tests** - Critical user workflows - **Component tests** - UI component testing (if requested) - **Fixtures** - Shared utilities and setup ### Component Testing by Framework TEA supports component testing using framework-appropriate tools: | Your Framework | Component Testing Tool | Tests Location | | -------------- | ------------------------------ | ----------------------------------------- | | **Cypress** | Cypress Component Testing | `tests/component/` | | **Playwright** | Vitest + React Testing Library | `tests/component/` or `src/**/*.test.tsx` | **Note:** Component tests use separate tooling from E2E tests: - Cypress users: TEA generates Cypress Component Tests - Playwright users: TEA generates Vitest + React Testing Library tests ### Quality Features - **Network-first patterns** - Wait for actual responses, not timeouts - **Deterministic tests** - No flakiness, no conditionals - **Self-cleaning** - Tests don't leave test data behind - **Parallel-safe** - Can run all tests concurrently ### Documentation - **Updated README** - How to run tests - **Test structure explanation** - Where tests live - **Definition of Done** - Quality standards ## Tips ### Start with Test Design Run `test-design` before `automate` for better results: ``` test-design # Risk assessment, priorities automate # Generate tests based on priorities ``` TEA will focus on P0/P1 scenarios and skip low-value tests. ### Prioritize Test Levels Not everything needs E2E tests: **Good strategy:** ``` - P0 scenarios: API + E2E tests - P1 scenarios: API tests only - P2 scenarios: API tests (happy path) - P3 scenarios: Skip or add later ``` **Why?** - API tests are 10x faster than E2E - API tests are more reliable (no browser flakiness) - E2E tests reserved for critical user journeys ### Avoid Duplicate Coverage Tell TEA about existing tests: ``` We already have tests in: - tests/e2e/profile-view.spec.ts (viewing profile) - tests/api/auth.spec.ts (authentication) Don't duplicate that coverage ``` TEA will analyze existing tests and only generate new scenarios. ### MCP Enhancements (Optional) If you have MCP servers configured (`tea_use_mcp_enhancements: true`), TEA can use them during `automate` for: - **Healing mode:** Fix broken selectors, update assertions, enhance with trace analysis - **Recording mode:** Verify selectors with live browser, capture network requests No prompts - TEA uses MCPs automatically when available. See [Enable MCP Enhancements](/docs/tea/how-to/customization/enable-tea-mcp-enhancements.md) for setup. ### Generate Tests Incrementally Don't generate all tests at once: **Iteration 1:** ``` Generate P0 tests only (critical path) Run: automate ``` **Iteration 2:** ``` Generate P1 tests (high value scenarios) Run: automate Tell TEA to avoid P0 coverage ``` **Iteration 3:** ``` Generate P2 tests (if time permits) Run: automate ``` This iterative approach: - Provides fast feedback - Allows validation before proceeding - Keeps test generation focused ## Common Issues ### Tests Pass But Coverage Is Incomplete **Problem:** Tests pass but don't cover all scenarios. **Cause:** TEA wasn't given complete context. **Solution:** Provide more details: ``` Generate tests for: - All acceptance criteria in story-profile.md - Error scenarios (validation, authorization) - Edge cases (empty fields, long inputs) ``` ### Too Many Tests Generated **Problem:** TEA generated 50 tests for a simple feature. **Cause:** Didn't specify priorities or scope. **Solution:** Be specific: ``` Generate ONLY: - P0 and P1 scenarios - API tests for all scenarios - E2E tests only for critical workflows - Skip P2/P3 for now ``` ### Tests Duplicate Existing Coverage **Problem:** New tests cover the same scenarios as existing tests. **Cause:** Didn't tell TEA about existing tests. **Solution:** Specify existing coverage: ``` We already have these tests: - tests/api/profile.spec.ts (GET /api/profile) - tests/e2e/profile-view.spec.ts (viewing profile) Generate tests for scenarios NOT covered by those files ``` ### MCP Enhancements for Better Selectors If you have MCP servers configured, TEA verifies selectors against live browser. Otherwise, TEA generates accessible selectors (`getByRole`, `getByLabel`) by default. Setup: Answer "Yes" to MCPs in BMad installer + configure MCP servers in your IDE. See [Enable MCP Enhancements](/docs/tea/how-to/customization/enable-tea-mcp-enhancements.md). ## Related Guides - [How to Run Test Design](/docs/tea/how-to/workflows/run-test-design.md) - Plan before generating - [How to Run ATDD](/docs/tea/how-to/workflows/run-atdd.md) - Failing tests before implementation - [How to Run Test Review](/docs/tea/how-to/workflows/run-test-review.md) - Audit generated quality ## Understanding the Concepts - [Testing as Engineering](/docs/tea/explanation/testing-as-engineering.md) - **Why TEA generates quality tests** (foundational) - [Risk-Based Testing](/docs/tea/explanation/risk-based-testing.md) - Why prioritize P0 over P3 - [Test Quality Standards](/docs/tea/explanation/test-quality-standards.md) - What makes tests good - [Fixture Architecture](/docs/tea/explanation/fixture-architecture.md) - Reusable test patterns ## Reference - [Command: *automate](/docs/tea/reference/commands.md#automate) - Full command reference - [TEA Configuration](/docs/tea/reference/configuration.md) - MCP and Playwright Utils options --- Generated with [BMad Method](https://bmad-method.org) - TEA (Test Architect)