diff --git a/src/bmm/workflows/4-implementation/create-story/checklist.md b/src/bmm/workflows/4-implementation/create-story/checklist.md index 6fc678994..bab1afb4e 100644 --- a/src/bmm/workflows/4-implementation/create-story/checklist.md +++ b/src/bmm/workflows/4-implementation/create-story/checklist.md @@ -35,7 +35,7 @@ This is a COMPETITION to create the **ULTIMATE story context** that makes LLM de - The `{project-root}/_bmad/core/tasks/validate-workflow.xml` framework will automatically: - Load this checklist file - - Load the newly created story file (`{story_file_path}`) + - Load the story file (`{story_file}` when provided, otherwise `{default_output_file}`) - Load workflow variables from `{installed_path}/workflow.yaml` - Execute the validation process @@ -51,7 +51,7 @@ This is a COMPETITION to create the **ULTIMATE story context** that makes LLM de - **Story file**: The story file to review and improve - **Workflow variables**: From workflow.yaml (implementation_artifacts, epics_file, etc.) - **Source documents**: Epics, architecture, etc. (discovered or provided) -- **Validation framework**: `validate-workflow.xml` (handles checklist execution) +- **Validation framework**: `validate-workflow.xml` (handles checklist execution and report generation) --- @@ -62,10 +62,20 @@ You will systematically re-do the entire story creation process, but with a crit ### **Step 1: Load and Understand the Target** 1. **Load the workflow configuration**: `{installed_path}/workflow.yaml` for variable inclusion -2. **Load the story file**: `{story_file_path}` (provided by user or discovered) +2. **Load the story file**: `{story_file}` first, fallback to `{default_output_file}` (or explicit `{document}` input) 3. **Load validation framework**: `{project-root}/_bmad/core/tasks/validate-workflow.xml` -4. **Extract metadata**: epic_num, story_num, story_key, story_title from story file -5. **Resolve all workflow variables**: implementation_artifacts, epics_file, architecture_file, etc. +4. **Resolve variables deterministically**: + - Load config_source file if present + - Parse workflow.yaml key/value pairs + - For any value matching `{config_source}:key`, resolve from the loaded config source + - Resolve system path variables (for example `{project-root}`, `{installed_path}`) in every path value + - Resolve system-generated values (for example `{date}`) using current execution context + - Required for this checklist flow: `{epics_file}`, `{architecture_file}`, `{implementation_artifacts}`, `{project-root}`, `{installed_path}`, and at least one story locator (`{story_file}` or `{default_output_file}`) + - Optional/fallback-capable values: validation `{checklist}` input and validation `{report}` input + - Validation task input contract: `workflow` is required; `checklist`, `document`, and `report` are optional with deterministic fallback + - Note: create-story invoke-task passes `document={default_output_file}` explicitly, which overrides fallback discovery + - If any required value remains unresolved, stop and request explicit user input before continuing +5. **Extract metadata**: epic_num, story_num, story_key, story_title from story file 6. **Understand current status**: What story implementation guidance is currently provided? **Note:** If running in fresh context, user should provide the story file path being reviewed. If running from create-story workflow, the validation framework will automatically discover the checklist and story file. diff --git a/src/bmm/workflows/4-implementation/create-story/instructions.xml b/src/bmm/workflows/4-implementation/create-story/instructions.xml index f9433371f..8ddc1c08c 100644 --- a/src/bmm/workflows/4-implementation/create-story/instructions.xml +++ b/src/bmm/workflows/4-implementation/create-story/instructions.xml @@ -280,21 +280,16 @@ testing_requirements - - previous_story_intelligence - + If previous story learnings are unavailable (for example this is story 1 in epic), set previous_story_intelligence to the canonical format: N/A: {one-line reason} + previous_story_intelligence - - git_intelligence_summary - + If git analysis is unavailable or not relevant, set git_intelligence_summary to the canonical format: N/A: {one-line reason} + git_intelligence_summary - - latest_tech_information - + If web research was not completed or not needed, set latest_tech_information to the canonical format: N/A: {one-line reason} + latest_tech_information - Validate against checklist at {installed_path}/checklist.md using _bmad/core/tasks/validate-workflow.xml Save story document unconditionally + Run {project-root}/_bmad/core/tasks/validate-workflow.xml with workflow={installed_path}/workflow.yaml checklist={installed_path}/checklist.md document={default_output_file} diff --git a/src/core/tasks/validate-workflow.xml b/src/core/tasks/validate-workflow.xml new file mode 100644 index 000000000..1463ddb70 --- /dev/null +++ b/src/core/tasks/validate-workflow.xml @@ -0,0 +1,179 @@ + + + Validate a generated document against checklist requirements with deterministic variable resolution and produce an actionable pass/fail report + + + + + + + + + + MANDATORY: Execute ALL steps in order. Do not skip any checklist item. + Always read COMPLETE files; do not sample with offsets. + If a file cannot be loaded in one read, read it in deterministic sequential chunks until full coverage is achieved and recorded. + Every non-N/A judgment must include concrete evidence from the document. + If a required path cannot be resolved, stop and ask for explicit user input. + Be strict and objective: no assumptions without evidence. + N/A is allowed only when an explicit conditional requirement is not applicable; never use N/A due to missing evidence. + + + + + Load workflow yaml from input {workflow} + Resolve variables in this order: + 1) load config_source file if present + 2) resolve all {config_source}:key references + 3) resolve system path variables ({project-root}, {installed_path}) + 4) resolve system-generated values (date) + + Determine checklist path in priority order: + - explicit input {checklist} + - workflow.yaml field "validation" + - sibling file checklist.md in workflow directory + + Determine document path in priority order: + - explicit input {document} + - resolved variable {story_file} if present + - resolved variable {default_output_file} if present + + Ask user: "Which document should I validate?" and WAIT + Normalize resolved workflow/checklist/document paths to absolute paths before loading files + + + + Load full checklist content (use chunked sequential reads only when needed for large files, and record covered ranges) + Load full target document content (use chunked sequential reads only when needed for large files, and record covered ranges) + Extract story metadata when available (epic_num, story_num, story_id, story_key, title) from filename, heading, or frontmatter + Parse checklist into ordered sections and atomic validation items; assign each item a stable id (section_index.item_index) + Compute parsed_item_count and expected_item_count (from raw checklist requirement markers such as - [ ], - [x], and requirement list lines) + HALT with error: "Checklist parse divergence too large (parsed={parsed_item_count}, expected~={expected_item_count}); fix checklist formatting before validation" + Determine critical checks from explicit signals only: + - item-level markers: [CRITICAL], critical:true, MUST FIX + - section-level markers: headings containing CRITICAL (case-insensitive), MUST FIX, MUST-FIX, or the exact phrase "Must Fix Before Proceeding" + - XML section attributes: critical="true" or critical="yes" + - do not infer criticality from generic keywords alone + + Detect conditional expressions in checklist items (for example: if/when/unless + variable references) + HALT with error: "Checklist is empty or unparsable" + Ask user to provide all missing referenced metadata fields (epic_num, story_num, story_id, story_key, title/story_title) and WAIT before evaluating checklist items + + + + For every checklist item, evaluate one of: PASS, PARTIAL, FAIL, N/A + Initialize counters to zero before evaluation: + - pass_count, partial_count, fail_count, na_count + - critical_fail_count, critical_partial_count + - applicable_count, total_item_count, processed_item_count + - total_section_count, processed_section_count + + For each item: + - restate requirement in one short sentence + - if item contains explicit condition (for example "If story_num > 1") and condition is false, mark N/A with the exact reason + - locate explicit evidence in document (include line references when possible) + - consider implied coverage only when explicit text is absent + - assign verdict and rationale + - if PARTIAL/FAIL, describe impact and a concrete fix + - update all counters immediately after each verdict + + Process sections in deterministic order and increment processed_section_count after each section completes + HALT with error: "Validation incomplete: one or more checklist sections were not processed" + HALT with error: "Validation incomplete: one or more checklist items were not processed" + Compute applicable_count = pass_count + partial_count + fail_count + Compute pass_percent using applicable_count (if 0, set pass_percent=0) + Set gate_warning: + - "No applicable checklist items were evaluated; manual confirmation required" if applicable_count == 0 + - "N/A" otherwise + + Set gate decision deterministically: + - NEEDS_REVIEW if applicable_count == 0 + - FAIL if critical_fail_count > 0 + - FAIL if critical_partial_count > 0 + - FAIL if fail_count > 0 + - PASS otherwise + + DO NOT SKIP ANY ITEM OR SECTION + + + + Generate timestamp values: + - timestamp_utc for filenames in YYYYMMDD-HHmmss (UTC) + - generated_at_utc for report display in ISO-8601 UTC + + Set report path: + - use explicit input {report} when provided + - else save to target document folder as validation-report-{timestamp_utc}.md + + Write report with the format below + HALT with error: "Validation report could not be saved" + + + # Validation Report + + - Document: {document} + - Checklist: {checklist} + - Workflow: {workflow} + - Date: {generated_at_utc} + + ## Summary + - Overall pass rate: {pass_count}/{applicable_count} ({pass_percent}%) + - Critical failures: {critical_fail_count} + - Critical partials: {critical_partial_count} + - Gate decision: {PASS|FAIL|NEEDS_REVIEW} + - Gate warning: {gate_warning} + + ## Section Results + ### {Section Name} + - PASS: {count} + - PARTIAL: {count} + - FAIL: {count} + - N/A: {count} + + For each checklist item: + - [MARK] {item} + - Evidence: {quote or line reference} + - Analysis: {why mark is correct} + - Action (if PARTIAL/FAIL): {specific remediation} + + ## Must Fix Before Proceeding + - {all critical FAIL and critical PARTIAL items} + + ## Should Improve + - {all non-critical FAIL and non-critical PARTIAL items} + + ## Final Recommendation + 1. {highest-priority fix} + 2. {second-priority fix} + 3. {third-priority fix} + + + + + Present concise summary with counts and gate decision + Provide report path + State clearly that workflow should not proceed until fixes are applied + Return control immediately and continue calling workflow execution + HALT and wait for user direction + + + + + HALT if workflow file cannot be loaded + HALT if checklist file cannot be loaded + HALT if target document cannot be determined after user prompt + HALT if any checklist section is skipped + HALT if validation report cannot be saved + + + + Never skip checklist items + Every PASS/PARTIAL/FAIL must have evidence + Use deterministic variable resolution before asking the user + Always save a validation report file + N/A is valid only for explicit conditional non-applicability + Criticality must come from explicit checklist markers or critical sections + +