fix(bmad-prd): validation report only on explicit analysis request
Reconciles a contradiction across SKILL.md, validation-render.md,
headless.md, and headless-schemas.md about when validation-report.{html,md}
gets written. Rule: a report file is only written when the user has
specifically asked for analysis — Validate intent, or a mid-session
"produce a report" request. The Finalize discipline pass during
Create/Update keeps findings in-conversation: autofix obvious issues,
ask on ambiguous ones, never write a file.
- SKILL.md: Finalize step 3 no longer renders a report; Validate intent
wording softened from "HTML report" to "validation report".
- references/validation-render.md: drops the severity-based conditional
for markdown emission. Script now always writes both HTML and MD
side-by-side when invoked; trigger gating happens upstream.
- assets/headless-schemas.md: drops the "may be omitted in interactive
mode" caveat; validation_report is required for Validate intent.
- scripts/render-validation-html.py: adds render_markdown_report()
emitting a severity-grouped markdown companion at output_path.with_suffix('.md').
Returns markdown path in the stdout JSON summary alongside HTML path.
This commit is contained in:
parent
2a952a9189
commit
6404153f94
|
|
@ -26,7 +26,7 @@ At the opening greeting, let the user know they can invoke the skills `bmad-part
|
|||
|
||||
**Update.** Reconcile an existing PRD with a change signal. Orient via source extractors (see `## Constraints` → Extract, don't ingest) against the PRD, addendum, `decision-log.md`, and original inputs — then run the `## Discovery` posture against the change signal. Surface conflicts with prior decisions before changing. If the change is fundamental, offer Create instead of patching. When changes are applied, proceed to `## Finalize`.
|
||||
|
||||
**Validate** (or *analyze*). Critique an existing PRD against `{workflow.validation_checklist}`. Standalone — does NOT enter `## Finalize`. Orient via source extractors against `decision-log.md` and any original inputs to give the validator context. Spawn the validator subagent against `prd.md` (and `addendum.md` if present); produce findings + HTML report per `references/validation-render.md`. Always offer to roll findings into an Update.
|
||||
**Validate** (or *analyze*). Critique an existing PRD against `{workflow.validation_checklist}`. Standalone — does NOT enter `## Finalize`. Orient via source extractors against `decision-log.md` and any original inputs to give the validator context. Spawn the validator subagent against `prd.md` (and `addendum.md` if present); produce findings and a validation report per `references/validation-render.md`. Always offer to roll findings into an Update.
|
||||
|
||||
## Discovery
|
||||
|
||||
|
|
@ -82,7 +82,7 @@ In both modes, resolve decisions conversationally rather than silently deferring
|
|||
|
||||
1. Decision log audit: walk `decision-log.md` with the user — each entry captured in PRD, in addendum, or set aside.
|
||||
2. Input reconciliation: subagent per user-supplied input against `prd.md` + `addendum.md`; surface gaps, especially qualitative ideas (tone, voice, feel) the FR structure silently drops. Must happen before polish.
|
||||
3. Discipline pass: validator subagent against `prd.md` with `{workflow.validation_checklist}`; findings + HTML report per `references/validation-render.md`. Resolve before polish.
|
||||
3. Discipline pass: validator subagent against `prd.md` with `{workflow.validation_checklist}`. Findings stay in-conversation — autofix obvious issues, ask on ambiguous ones. No report file is written. Resolve before polish.
|
||||
4. Open-items review: triage all Open Questions, `[ASSUMPTION]` tags, and `[NOTE FOR PM]` callouts. Surface only phase-blockers one at a time; resolve before calling the PRD ready. Log deferred items to `decision-log.md`. If phase-blocking count is high, flag it.
|
||||
5. Polish: apply `{workflow.doc_standards}` to `prd.md` and `addendum.md` via parallel subagents.
|
||||
6. External handoffs: execute `{workflow.external_handoffs}` entries; surface returned URLs/IDs. Skip and flag unavailable tools.
|
||||
|
|
|
|||
|
|
@ -58,7 +58,7 @@ Every headless run ends with one of these payloads. Omit keys for artifacts not
|
|||
}
|
||||
```
|
||||
|
||||
`validation_report` is omitted for runs with no substantive findings (fewer than ~5 issues, none Critical, interactive mode); findings surface inline only.
|
||||
`validation_report` is always written for Validate intent — the path here is required, not optional.
|
||||
|
||||
## Blocked
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Validation Rendering
|
||||
|
||||
How the validator subagent's findings become the HTML report. Loaded by the parent on any Validate or Finalize-step-3 invocation.
|
||||
How the validator subagent's findings become a validation report. Loaded only when the user has explicitly asked for analysis — either Validate intent or a mid-session report request. The Finalize discipline pass during Create/Update does NOT render a report; its findings stay in-conversation.
|
||||
|
||||
## Validator subagent output contract
|
||||
|
||||
|
|
@ -53,8 +53,6 @@ python3 {skill-root}/scripts/render-validation-html.py \
|
|||
|
||||
Include `--open` for interactive runs (auto-opens in default browser). Omit `--open` in headless runs.
|
||||
|
||||
The script computes pass/warn/fail/na counts, derives a grade (Excellent / Good / Fair / Poor) from critical-fail and total-fail counts, renders an inline SVG score bar, groups findings by category, and substitutes into the template. Returns a one-line JSON summary on stdout: `{"output": "...", "grade": "...", "stats": {...}}`.
|
||||
The script writes two artifacts side-by-side: the HTML report at `--output`, and a markdown companion at the same path with `.md` extension (e.g. `validation-report.md`). Both are always produced when the script runs — trigger gating happens upstream (the script is only invoked when the user has asked for analysis). It computes pass/warn/fail/na counts, derives a grade (Excellent / Good / Fair / Poor) from critical-fail and total-fail counts, renders an inline SVG score bar in the HTML, groups findings by category, and returns a one-line JSON summary on stdout: `{"output": "...", "markdown": "...", "grade": "...", "stats": {...}}`.
|
||||
|
||||
## Markdown companion
|
||||
|
||||
When findings include any Critical-severity item or >5 total fail/warn items, also write `{doc_workspace}/validation-report.md` — a markdown rendering of the same findings, grouped by severity, with PRD line references. Update mode consumes the markdown form cleanly when rolling findings into a revision.
|
||||
Re-running validation overwrites the existing report files in place. Markdown form is what Update mode reads when rolling findings into a revision.
|
||||
|
|
|
|||
|
|
@ -2,12 +2,13 @@
|
|||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# ///
|
||||
"""Render a PRD validation findings JSON into a styled HTML report.
|
||||
"""Render a PRD validation findings JSON into HTML + markdown reports.
|
||||
|
||||
Reads structured findings produced by the validator subagent, groups them by
|
||||
category (explicit `category` field, else derived from ID prefix), computes a
|
||||
pass/warn/fail summary and grade, substitutes into the configured HTML
|
||||
template, and optionally opens the result in the default browser.
|
||||
template, writes a markdown companion at the same path with `.md` extension,
|
||||
and optionally opens the HTML in the default browser.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
|
|
@ -134,6 +135,78 @@ def render_category(name: str, findings: list[dict]) -> str:
|
|||
)
|
||||
|
||||
|
||||
SEVERITY_ORDER = ["critical", "high", "medium", "low"]
|
||||
|
||||
|
||||
def render_finding_md(f: dict) -> str:
|
||||
status = (f.get("status") or "n/a").upper()
|
||||
severity = (f.get("severity") or "low").lower()
|
||||
fid = f.get("id") or ""
|
||||
title = f.get("title") or fid
|
||||
location = f.get("location") or ""
|
||||
note = f.get("note") or ""
|
||||
fix = f.get("suggested_fix") or ""
|
||||
|
||||
lines = [f"### [{status}] {fid} — {title} _(severity: {severity})_"]
|
||||
if location:
|
||||
lines.append(f"- **Location:** {location}")
|
||||
if note:
|
||||
lines.append(f"- **Finding:** {note}")
|
||||
if fix:
|
||||
lines.append(f"- **Suggested fix:** {fix}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def render_markdown_report(data: dict, findings: list[dict], stats: dict, grade: str) -> str:
|
||||
prd_name = data.get("prd_name") or "PRD"
|
||||
prd_path = data.get("prd_path") or ""
|
||||
checklist_path = data.get("checklist_path") or ""
|
||||
timestamp = data.get("timestamp") or datetime.now().isoformat(timespec="seconds")
|
||||
synthesis = data.get("overall_synthesis") or ""
|
||||
|
||||
out = [
|
||||
f"# Validation Report — {prd_name}",
|
||||
"",
|
||||
f"- **PRD:** `{prd_path}`",
|
||||
f"- **Checklist:** `{checklist_path}`",
|
||||
f"- **Run at:** {timestamp}",
|
||||
f"- **Grade:** {grade}",
|
||||
"",
|
||||
f"**Summary:** {stats['passed']} pass · {stats['warned']} warn · {stats['failed']} fail · {stats['na']} n/a "
|
||||
f"(total {stats['total']}; critical fails: {stats['failed_critical']}, high fails: {stats['failed_high']})",
|
||||
]
|
||||
if synthesis:
|
||||
out += ["", "## Overall synthesis", "", synthesis]
|
||||
|
||||
# Group by severity then status: failed criticals first, then highs, etc.
|
||||
by_sev: dict[str, list[dict]] = {s: [] for s in SEVERITY_ORDER}
|
||||
other: list[dict] = []
|
||||
for f in findings:
|
||||
sev = (f.get("severity") or "low").lower()
|
||||
if sev in by_sev:
|
||||
by_sev[sev].append(f)
|
||||
else:
|
||||
other.append(f)
|
||||
|
||||
out += ["", "## Findings by severity"]
|
||||
any_findings = False
|
||||
for sev in SEVERITY_ORDER:
|
||||
items = by_sev[sev]
|
||||
if not items:
|
||||
continue
|
||||
any_findings = True
|
||||
out += ["", f"### {sev.capitalize()} ({len(items)})", ""]
|
||||
out += [render_finding_md(f) for f in items]
|
||||
if other:
|
||||
any_findings = True
|
||||
out += ["", f"### Other ({len(other)})", ""]
|
||||
out += [render_finding_md(f) for f in other]
|
||||
if not any_findings:
|
||||
out += ["", "_No findings._"]
|
||||
|
||||
return "\n".join(out) + "\n"
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(description="Render PRD validation findings to HTML.")
|
||||
parser.add_argument("--findings", required=True, help="Path to validation-findings.json")
|
||||
|
|
@ -186,7 +259,15 @@ def main(argv: list[str]) -> int:
|
|||
rendered = string.Template(template).safe_substitute(substitutions)
|
||||
output_path.write_text(rendered)
|
||||
|
||||
print(json.dumps({"output": str(output_path), "grade": grade, "stats": stats}))
|
||||
md_path = output_path.with_suffix(".md")
|
||||
md_path.write_text(render_markdown_report(data, findings, stats, grade))
|
||||
|
||||
print(json.dumps({
|
||||
"output": str(output_path),
|
||||
"markdown": str(md_path),
|
||||
"grade": grade,
|
||||
"stats": stats,
|
||||
}))
|
||||
|
||||
if args.open:
|
||||
webbrowser.open(output_path.resolve().as_uri())
|
||||
|
|
|
|||
Loading…
Reference in New Issue