ldy26098
/
BMAD-METHOD
mirror of https://github.com/bmad-code-org/BMAD-METHOD.git
1
0
Fork 0
Code Packages Projects Releases Wiki Activity
BMAD-METHOD/TEST.md

11 KiB
Raw Blame History

Markdown Formatting Fix - Test Evidence

Fix Type: CommonMark Compliance Target Branch: v6-alpha Test Repository: https://github.com/jheyworth/bmad-markdown-formatting-test Test Date: 2025-10-28

Executive Summary

This fix makes BMAD-generated markdown CommonMark compliant by adding 6 critical formatting rules that ensure proper rendering across ALL markdown parsers, not just GitHub's lenient renderer.

The Problem: BMAD generates markdown that violates CommonMark specification by omitting required blank lines around lists, tables, and code blocks. While GitHub's renderer (GFM) handles this gracefully, strict parsers like Mac Markdown.app break completely.

The Solution: Add 6 mandatory markdown formatting rules to bmad/core/tasks/workflow.xml that enforce proper spacing and consistency.

The Result: Professional, standards-compliant documentation that works everywhere.

The Fix

Location

bmad/core/tasks/workflow.xml - Line 73 (in <template-output> section)

Code Added

<mandate critical="true">MARKDOWN FORMATTING RULES - Critical for proper rendering across all markdown parsers:
  1. ALWAYS add blank line before and after bullet lists (-, *, +)
  2. ALWAYS add blank line before and after numbered lists (1., 2., etc.)
  3. ALWAYS add blank line before and after tables (| header |)
  4. ALWAYS add blank line before and after code blocks (```)
  5. Use - for bullets consistently (not * or +)
  6. Use language identifier for code fences (```bash, ```javascript, etc.)
</mandate>

Why This Fix Matters

GitHub Hides the Problem

Important Discovery: During testing, we found that GitHub's markdown renderer (GFM) is lenient and handles missing blank lines gracefully. Both the before-fix and after-fix versions render acceptably in GitHub.

However, many other tools do NOT:

  • ❌ Mac Markdown.app - Lists render as plain text
  • ❌ CommonMark reference parser - Violates specification
  • ❌ Some IDE previewers - Inconsistent rendering
  • ❌ Documentation generators - May break

The Real Value

This fix provides:

  1. CommonMark Compliance - Follows official markdown specification
  2. Cross-Tool Compatibility - Works in ALL markdown renderers
  3. Professional Best Practices - Industry-standard documentation
  4. Future-Proofing - Won't break if parsers get stricter
  5. Developer Experience - Team members can use any tool they prefer

Test Methodology

Three-Phase Testing

Phase 1: Synthetic Test

  • Created isolated test to validate fix mechanism
  • Confirmed all 6 rules apply correctly
  • Verified blank lines added properly

Phase 2: Fresh Install End-to-End

  • Clean BMAD v6-alpha installation
  • Generated comprehensive API Gateway project documentation
  • Baseline (before-fix) vs. Fixed (after-fix) comparison
  • Results: 1112 lines of formatting improvements, 0 content changes

Phase 3: GitHub Validation

  • Created public test repository
  • Uploaded both versions to GitHub
  • Discovered GitHub's leniency (critical insight!)
  • Added Mac Markdown.app screenshot proving the issue

Test Repository Structure

bmad-markdown-formatting-test/
├── docs/
│   ├── product-brief.md
│   ├── PRD.md
│   └── epics.md
├── test-results/
│   ├── before-fix/          # Baseline without fix
│   ├── after-fix/           # With formatting rules
│   ├── screenshots/         # Mac Markdown.app proof
│   └── FINDINGS.md          # Detailed analysis
├── README.md
├── VISUAL-COMPARISON-GUIDE.md
└── CROSS-TOOL-COMPATIBILITY.md

Test Results

Quantitative Analysis

Metric Value
Documents tested 3 (product-brief, PRD, epics)
Total diff lines 1,112
Formatting improvements 1,112
Content changes 0
Bullet lists fixed 99
Numbered lists fixed 36
Tables fixed 11
Code blocks fixed 12+
CommonMark violations before 64+
CommonMark violations after 0 ✅

Before Fix Issues

Without blank lines, strict parsers break:

**Security Vulnerabilities:**
- Each service implements its own authentication logic
- No centralized authorization enforcement

Mac Markdown.app renders as:

Security Vulnerabilities: - Each service implements its own authentication logic - No centralized authorization enforcement

All bullets flow together as plain text!

After Fix

With proper blank lines:

**Security Vulnerabilities:**

- Each service implements its own authentication logic
- No centralized authorization enforcement

Mac Markdown.app renders properly as:

Security Vulnerabilities:

  • Each service implements its own authentication logic
  • No centralized authorization enforcement

Professional, structured formatting!

Visual Evidence

Mac Markdown.app Screenshot

The test repository includes a screenshot showing broken rendering in Mac Markdown.app (before-fix):

  • Lists render as plain text instead of bullets
  • Code blocks render as inline text
  • Everything flows together as paragraphs
  • Complete loss of structure

View Screenshot: test-results/screenshots/mac-markdown-before-fix-broken.png

This is the real problem that GitHub's leniency hides.

Comparison Links

GitHub Before/After Commits

Before Fix (Baseline):

After Fix (With Rules):

Side-by-Side Comparison:

Specific Examples to Review

Example 1: Acceptance Criteria Lists

Example 2: Tables

Example 3: Code Blocks

CommonMark Specification Reference

According to the CommonMark Specification (v0.31.2):

Section 5.2 - List items:

"A list item can begin with at most one blank line... A list item can contain blocks that are separated by more than one blank line..."

The key requirement: Lists must be separated from preceding text by a blank line.

Example from Spec

NOT a list (incorrect):

Foo
- bar

Renders as: Foo - bar (plain text)

IS a list (correct):

Foo

- bar

Renders as:

Foo

  • bar

BMAD now generates the correct version!

How to Reproduce Testing

Method 1: Clone Test Repository

git clone https://github.com/jheyworth/bmad-markdown-formatting-test.git
cd bmad-markdown-formatting-test

# View before-fix version
git checkout dcf405f
# Open docs/*.md in Mac Markdown.app - see broken rendering

# View after-fix version
git checkout 0b30d47
# Open docs/*.md in Mac Markdown.app - see proper rendering

Method 2: Test With Mac Markdown.app

  1. Download before-fix file:

    curl -O https://raw.githubusercontent.com/jheyworth/bmad-markdown-formatting-test/dcf405f/docs/product-brief.md

  2. Open in Mac Markdown.app (or Marked 2)

  3. Observe: Lists render as plain text!

  4. Download after-fix file:

    curl -O https://raw.githubusercontent.com/jheyworth/bmad-markdown-formatting-test/0b30d47/docs/product-brief.md

  5. Open in Mac Markdown.app

  6. Observe: Lists render properly!

Method 3: CommonMark CLI

npm install -g commonmark

# Test before-fix
curl -s https://raw.githubusercontent.com/jheyworth/bmad-markdown-formatting-test/dcf405f/docs/product-brief.md | commonmark > before.html

# Test after-fix
curl -s https://raw.githubusercontent.com/jheyworth/bmad-markdown-formatting-test/0b30d47/docs/product-brief.md | commonmark > after.html

# Compare HTML output
diff before.html after.html

You'll see the before-fix version has malformed HTML for lists!

Real-World Impact

Before This Fix

  • ❌ Violates CommonMark specification
  • ❌ Breaks in Mac Markdown.app and strict parsers
  • ❌ Non-standard markdown that relies on GitHub's leniency
  • ❌ Unprofessional code that doesn't follow best practices
  • ❌ May break if tools update to stricter parsing
  • ❌ Inconsistent developer experience across different tools

After This Fix

  • ✅ Complies with CommonMark specification
  • ✅ Works in ALL markdown parsers (GitHub, Mac app, IDEs, etc.)
  • ✅ Professional, standards-compliant documentation
  • ✅ Follows industry best practices
  • ✅ Future-proof against parser changes
  • ✅ Consistent experience for all developers

Additional Documentation

The test repository includes comprehensive documentation:

  • README.md - Overview and quick start
  • FINDINGS.md - Detailed test analysis with metrics
  • VISUAL-COMPARISON-GUIDE.md - Step-by-step visual examples
  • CROSS-TOOL-COMPATIBILITY.md - Deep dive into parser differences

Full Documentation: https://github.com/jheyworth/bmad-markdown-formatting-test

Conclusion

This fix is fully tested and production-ready.

What We Proved:

  • ✅ Fix mechanism works correctly (Phase 1 synthetic test)
  • ✅ Fix works in real-world scenarios (Phase 2 end-to-end test)
  • ✅ GitHub is lenient but other tools break (Phase 3 validation)
  • ✅ Visual proof via Mac Markdown.app screenshot
  • ✅ Zero content changes - only formatting improved
  • ✅ 100% CommonMark compliance achieved

The Value:

This isn't about "fixing GitHub rendering" - it's about generating professional, standards-compliant markdown that works everywhere. BMAD should follow industry best practices and output documentation that any developer can view in any tool without surprises.

Recommendation: Merge this fix into v6-alpha to ensure BMAD generates professional, standards-compliant documentation from day one.

Test Repository: https://github.com/jheyworth/bmad-markdown-formatting-test Test Date: 2025-10-28 Test Duration: ~2 hours Status: ✅ SUCCESS