From 9e4e37666f4538fa797c8d73624031f90d83b581 Mon Sep 17 00:00:00 2001 From: Maple Date: Sun, 15 Feb 2026 01:04:24 +0800 Subject: [PATCH] fix(create-story): address validator determinism and bot feedback --- .../create-story/checklist.md | 4 +- .../create-story/instructions.xml | 7 ++- src/core/tasks/validate-workflow.xml | 50 +++++++++++++++---- 3 files changed, 45 insertions(+), 16 deletions(-) diff --git a/src/bmm/workflows/4-implementation/create-story/checklist.md b/src/bmm/workflows/4-implementation/create-story/checklist.md index 1fe7c2d5c..545b0a7a1 100644 --- a/src/bmm/workflows/4-implementation/create-story/checklist.md +++ b/src/bmm/workflows/4-implementation/create-story/checklist.md @@ -35,7 +35,7 @@ This is a COMPETITION to create the **ULTIMATE story context** that makes LLM de - The `{project-root}/_bmad/core/tasks/validate-workflow.xml` framework will automatically: - Load this checklist file - - Load the newly created story file (`{story_file_path}`) + - Load the newly created story file (`{default_output_file}`) - Load workflow variables from `{installed_path}/workflow.yaml` - Execute the validation process @@ -62,7 +62,7 @@ You will systematically re-do the entire story creation process, but with a crit ### **Step 1: Load and Understand the Target** 1. **Load the workflow configuration**: `{installed_path}/workflow.yaml` for variable inclusion -2. **Load the story file**: `{story_file_path}` (provided by user or discovered) +2. **Load the story file**: `{default_output_file}` (or explicit `{document}` input) 3. **Load validation framework**: `{project-root}/_bmad/core/tasks/validate-workflow.xml` 4. **Resolve variables deterministically**: - Parse workflow.yaml key/value pairs diff --git a/src/bmm/workflows/4-implementation/create-story/instructions.xml b/src/bmm/workflows/4-implementation/create-story/instructions.xml index 2f67a9004..0f3ef5b78 100644 --- a/src/bmm/workflows/4-implementation/create-story/instructions.xml +++ b/src/bmm/workflows/4-implementation/create-story/instructions.xml @@ -288,9 +288,8 @@ git_intelligence_summary - - latest_tech_information - + If web research was not completed or not needed, set latest_tech_information to an explicit N/A note with reason + latest_tech_information - Validate against checklist at {installed_path}/checklist.md using _bmad/core/tasks/validate-workflow.xml and target file {default_output_file} + Run _bmad/core/tasks/validate-workflow.xml with workflow={installed_path}/workflow.yaml checklist={installed_path}/checklist.md document={default_output_file} Save story document unconditionally diff --git a/src/core/tasks/validate-workflow.xml b/src/core/tasks/validate-workflow.xml index 7d6edccec..7db351faf 100644 --- a/src/core/tasks/validate-workflow.xml +++ b/src/core/tasks/validate-workflow.xml @@ -7,16 +7,18 @@ - - + + MANDATORY: Execute ALL steps in order. Do not skip any checklist item. Always read COMPLETE files; do not sample with offsets. + If a file cannot be loaded in one read, read it in deterministic sequential chunks until full coverage is achieved and recorded. Every non-N/A judgment must include concrete evidence from the document. If a required path cannot be resolved, stop and ask for explicit user input. Be strict and objective: no assumptions without evidence. + N/A is allowed only when an explicit conditional requirement is not applicable; never use N/A due to missing evidence. @@ -38,36 +40,61 @@ - resolved variable {story_file} if present - resolved variable {default_output_file} if present - Try fuzzy discovery in implementation_artifacts: pick most likely recent .md output and state this inference explicitly Ask user: "Which document should I validate?" and WAIT + Normalize resolved workflow/checklist/document paths to absolute paths before loading files - Load full checklist content - Load full target document content - Extract story metadata when available (epic_num, story_num, story_id) from document title or filename for conditional checks - Parse checklist into sections and atomic validation items - Mark items containing terms like "critical", "must", "required", "blocking" as critical checks + Load full checklist content (use chunked sequential reads only when needed for large files, and record covered ranges) + Load full target document content (use chunked sequential reads only when needed for large files, and record covered ranges) + Extract story metadata when available (epic_num, story_num, story_id, story_key, title) from filename, heading, or frontmatter + Parse checklist into ordered sections and atomic validation items; assign each item a stable id (section_index.item_index) + Determine critical checks from explicit signals only: item-level markers ([CRITICAL], critical:true, MUST FIX) or critical section labels; do not infer criticality from generic keywords alone + Detect conditional expressions in checklist items (for example: if/when/unless + variable references) HALT with error: "Checklist is empty or unparsable" + Record metadata_gap=true and list missing metadata fields for explicit PARTIAL decisions in step 3 For every checklist item, evaluate one of: PASS, PARTIAL, FAIL, N/A + Initialize counters to zero before evaluation: + - pass_count, partial_count, fail_count, na_count + - critical_fail_count, critical_partial_count + - applicable_count, total_item_count, processed_item_count + - total_section_count, processed_section_count + For each item: - restate requirement in one short sentence - if item contains explicit condition (for example "If story_num > 1") and condition is false, mark N/A with the exact reason + - if item condition depends on missing metadata, mark PARTIAL (not N/A) and specify required metadata - locate explicit evidence in document (include line references when possible) - consider implied coverage only when explicit text is absent - assign verdict and rationale - if PARTIAL/FAIL, describe impact and a concrete fix + - update all counters immediately after each verdict + + Process sections in deterministic order and increment processed_section_count after each section completes + HALT with error: "Validation incomplete: one or more checklist sections were not processed" + HALT with error: "Validation incomplete: one or more checklist items were not processed" + Compute applicable_count = pass_count + partial_count + fail_count + Compute pass_percent using applicable_count (if 0, set pass_percent=0) + Set gate decision deterministically: + - FAIL if critical_fail_count > 0 + - FAIL if critical_partial_count > 0 + - FAIL if fail_count > 0 + - PASS otherwise DO NOT SKIP ANY ITEM OR SECTION + Generate timestamp values: + - timestamp_utc for filenames in YYYYMMDD-HHmmss (UTC) + - generated_at_utc for report display in ISO-8601 UTC + Set report path: - use explicit input {report} when provided - - else save to target document folder as validation-report-{timestamp}.md + - else save to target document folder as validation-report-{timestamp_utc}.md Write report with the format below @@ -77,7 +104,7 @@ - Document: {document} - Checklist: {checklist} - Workflow: {workflow} - - Date: {timestamp} + - Date: {generated_at_utc} ## Summary - Overall pass rate: {pass_count}/{applicable_count} ({pass_percent}%) @@ -114,6 +141,7 @@ Present concise summary with counts and gate decision Provide report path + Call out missing metadata fields that caused PARTIAL results and how to supply them State clearly that workflow should not proceed until fixes are applied HALT and wait for user direction @@ -131,5 +159,7 @@ Every PASS/PARTIAL/FAIL must have evidence Use deterministic variable resolution before asking the user Always save a validation report file + N/A is valid only for explicit conditional non-applicability + Criticality must come from explicit checklist markers or critical sections