fix(bmad-prd): validation report only on explicit analysis request

Reconciles a contradiction across SKILL.md, validation-render.md,
headless.md, and headless-schemas.md about when validation-report.{html,md}
gets written. Rule: a report file is only written when the user has
specifically asked for analysis — Validate intent, or a mid-session
"produce a report" request. The Finalize discipline pass during
Create/Update keeps findings in-conversation: autofix obvious issues,
ask on ambiguous ones, never write a file.

- SKILL.md: Finalize step 3 no longer renders a report; Validate intent
  wording softened from "HTML report" to "validation report".
- references/validation-render.md: drops the severity-based conditional
  for markdown emission. Script now always writes both HTML and MD
  side-by-side when invoked; trigger gating happens upstream.
- assets/headless-schemas.md: drops the "may be omitted in interactive
  mode" caveat; validation_report is required for Validate intent.
- scripts/render-validation-html.py: adds render_markdown_report()
  emitting a severity-grouped markdown companion at output_path.with_suffix('.md').
  Returns markdown path in the stdout JSON summary alongside HTML path.
This commit is contained in:
Brian Madison 2026-05-13 07:47:58 -05:00
parent 2a952a9189
commit 6404153f94
4 changed files with 90 additions and 11 deletions

View File

@ -26,7 +26,7 @@ At the opening greeting, let the user know they can invoke the skills `bmad-part
**Update.** Reconcile an existing PRD with a change signal. Orient via source extractors (see `## Constraints` → Extract, don't ingest) against the PRD, addendum, `decision-log.md`, and original inputs — then run the `## Discovery` posture against the change signal. Surface conflicts with prior decisions before changing. If the change is fundamental, offer Create instead of patching. When changes are applied, proceed to `## Finalize`.
**Validate** (or *analyze*). Critique an existing PRD against `{workflow.validation_checklist}`. Standalone — does NOT enter `## Finalize`. Orient via source extractors against `decision-log.md` and any original inputs to give the validator context. Spawn the validator subagent against `prd.md` (and `addendum.md` if present); produce findings + HTML report per `references/validation-render.md`. Always offer to roll findings into an Update.
**Validate** (or *analyze*). Critique an existing PRD against `{workflow.validation_checklist}`. Standalone — does NOT enter `## Finalize`. Orient via source extractors against `decision-log.md` and any original inputs to give the validator context. Spawn the validator subagent against `prd.md` (and `addendum.md` if present); produce findings and a validation report per `references/validation-render.md`. Always offer to roll findings into an Update.
## Discovery
@ -82,7 +82,7 @@ In both modes, resolve decisions conversationally rather than silently deferring
1. Decision log audit: walk `decision-log.md` with the user — each entry captured in PRD, in addendum, or set aside.
2. Input reconciliation: subagent per user-supplied input against `prd.md` + `addendum.md`; surface gaps, especially qualitative ideas (tone, voice, feel) the FR structure silently drops. Must happen before polish.
3. Discipline pass: validator subagent against `prd.md` with `{workflow.validation_checklist}`; findings + HTML report per `references/validation-render.md`. Resolve before polish.
3. Discipline pass: validator subagent against `prd.md` with `{workflow.validation_checklist}`. Findings stay in-conversation — autofix obvious issues, ask on ambiguous ones. No report file is written. Resolve before polish.
4. Open-items review: triage all Open Questions, `[ASSUMPTION]` tags, and `[NOTE FOR PM]` callouts. Surface only phase-blockers one at a time; resolve before calling the PRD ready. Log deferred items to `decision-log.md`. If phase-blocking count is high, flag it.
5. Polish: apply `{workflow.doc_standards}` to `prd.md` and `addendum.md` via parallel subagents.
6. External handoffs: execute `{workflow.external_handoffs}` entries; surface returned URLs/IDs. Skip and flag unavailable tools.

View File

@ -58,7 +58,7 @@ Every headless run ends with one of these payloads. Omit keys for artifacts not
}
```
`validation_report` is omitted for runs with no substantive findings (fewer than ~5 issues, none Critical, interactive mode); findings surface inline only.
`validation_report` is always written for Validate intent — the path here is required, not optional.
## Blocked

View File

@ -1,6 +1,6 @@
# Validation Rendering
How the validator subagent's findings become the HTML report. Loaded by the parent on any Validate or Finalize-step-3 invocation.
How the validator subagent's findings become a validation report. Loaded only when the user has explicitly asked for analysis — either Validate intent or a mid-session report request. The Finalize discipline pass during Create/Update does NOT render a report; its findings stay in-conversation.
## Validator subagent output contract
@ -53,8 +53,6 @@ python3 {skill-root}/scripts/render-validation-html.py \
Include `--open` for interactive runs (auto-opens in default browser). Omit `--open` in headless runs.
The script computes pass/warn/fail/na counts, derives a grade (Excellent / Good / Fair / Poor) from critical-fail and total-fail counts, renders an inline SVG score bar, groups findings by category, and substitutes into the template. Returns a one-line JSON summary on stdout: `{"output": "...", "grade": "...", "stats": {...}}`.
The script writes two artifacts side-by-side: the HTML report at `--output`, and a markdown companion at the same path with `.md` extension (e.g. `validation-report.md`). Both are always produced when the script runs — trigger gating happens upstream (the script is only invoked when the user has asked for analysis). It computes pass/warn/fail/na counts, derives a grade (Excellent / Good / Fair / Poor) from critical-fail and total-fail counts, renders an inline SVG score bar in the HTML, groups findings by category, and returns a one-line JSON summary on stdout: `{"output": "...", "markdown": "...", "grade": "...", "stats": {...}}`.
## Markdown companion
When findings include any Critical-severity item or >5 total fail/warn items, also write `{doc_workspace}/validation-report.md` — a markdown rendering of the same findings, grouped by severity, with PRD line references. Update mode consumes the markdown form cleanly when rolling findings into a revision.
Re-running validation overwrites the existing report files in place. Markdown form is what Update mode reads when rolling findings into a revision.

View File

@ -2,12 +2,13 @@
# /// script
# requires-python = ">=3.10"
# ///
"""Render a PRD validation findings JSON into a styled HTML report.
"""Render a PRD validation findings JSON into HTML + markdown reports.
Reads structured findings produced by the validator subagent, groups them by
category (explicit `category` field, else derived from ID prefix), computes a
pass/warn/fail summary and grade, substitutes into the configured HTML
template, and optionally opens the result in the default browser.
template, writes a markdown companion at the same path with `.md` extension,
and optionally opens the HTML in the default browser.
"""
import argparse
@ -134,6 +135,78 @@ def render_category(name: str, findings: list[dict]) -> str:
)
SEVERITY_ORDER = ["critical", "high", "medium", "low"]
def render_finding_md(f: dict) -> str:
status = (f.get("status") or "n/a").upper()
severity = (f.get("severity") or "low").lower()
fid = f.get("id") or ""
title = f.get("title") or fid
location = f.get("location") or ""
note = f.get("note") or ""
fix = f.get("suggested_fix") or ""
lines = [f"### [{status}] {fid}{title} _(severity: {severity})_"]
if location:
lines.append(f"- **Location:** {location}")
if note:
lines.append(f"- **Finding:** {note}")
if fix:
lines.append(f"- **Suggested fix:** {fix}")
return "\n".join(lines)
def render_markdown_report(data: dict, findings: list[dict], stats: dict, grade: str) -> str:
prd_name = data.get("prd_name") or "PRD"
prd_path = data.get("prd_path") or ""
checklist_path = data.get("checklist_path") or ""
timestamp = data.get("timestamp") or datetime.now().isoformat(timespec="seconds")
synthesis = data.get("overall_synthesis") or ""
out = [
f"# Validation Report — {prd_name}",
"",
f"- **PRD:** `{prd_path}`",
f"- **Checklist:** `{checklist_path}`",
f"- **Run at:** {timestamp}",
f"- **Grade:** {grade}",
"",
f"**Summary:** {stats['passed']} pass · {stats['warned']} warn · {stats['failed']} fail · {stats['na']} n/a "
f"(total {stats['total']}; critical fails: {stats['failed_critical']}, high fails: {stats['failed_high']})",
]
if synthesis:
out += ["", "## Overall synthesis", "", synthesis]
# Group by severity then status: failed criticals first, then highs, etc.
by_sev: dict[str, list[dict]] = {s: [] for s in SEVERITY_ORDER}
other: list[dict] = []
for f in findings:
sev = (f.get("severity") or "low").lower()
if sev in by_sev:
by_sev[sev].append(f)
else:
other.append(f)
out += ["", "## Findings by severity"]
any_findings = False
for sev in SEVERITY_ORDER:
items = by_sev[sev]
if not items:
continue
any_findings = True
out += ["", f"### {sev.capitalize()} ({len(items)})", ""]
out += [render_finding_md(f) for f in items]
if other:
any_findings = True
out += ["", f"### Other ({len(other)})", ""]
out += [render_finding_md(f) for f in other]
if not any_findings:
out += ["", "_No findings._"]
return "\n".join(out) + "\n"
def main(argv: list[str]) -> int:
parser = argparse.ArgumentParser(description="Render PRD validation findings to HTML.")
parser.add_argument("--findings", required=True, help="Path to validation-findings.json")
@ -186,7 +259,15 @@ def main(argv: list[str]) -> int:
rendered = string.Template(template).safe_substitute(substitutions)
output_path.write_text(rendered)
print(json.dumps({"output": str(output_path), "grade": grade, "stats": stats}))
md_path = output_path.with_suffix(".md")
md_path.write_text(render_markdown_report(data, findings, stats, grade))
print(json.dumps({
"output": str(output_path),
"markdown": str(md_path),
"grade": grade,
"stats": stats,
}))
if args.open:
webbrowser.open(output_path.resolve().as_uri())