Skip to content

fix(docs): render fenced code at the end of a document - #969

Closed
rudycelekli wants to merge 2 commits into
openclaw:mainfrom
rudycelekli:fix/peekaboo-docs-eof-fence-20261005
Closed

rudycelekli wants to merge 2 commits into
openclaw:mainfrom
rudycelekli:fix/peekaboo-docs-eof-fence-20261005

Conversation

@rudycelekli

Copy link
Copy Markdown
Contributor

Summary

The docs builder accumulated fenced code until an explicit closing fence. If the document ended first, it discarded every captured code line and still reported a successful build.

Flush the pending fence at EOF using the existing syntax highlighter. Markdown permits the document end to close a fenced block; see CommonMark fenced code rules. Add EOF and explicitly closed controls, and include the new regression in test:docs-site/test:safe.

Observed production behavior

Actual builder entrypoint, owned Markdown fixture only. The shipped docs tree was also built without source changes.

Actual docs build entrypoint; owned EOF-terminated Markdown fixture.
before builder exit: 0
Rendered code blocks: 0
Copyable command present: False
after builder exit: 0
Rendered code blocks: 1
Copyable command present: True
Unmodified shipped docs build exit: 0
HTML pages emitted: 70

Validation

  • node --test tests/docs-site-eof-fence.test.mjs tests/docs-site-toc.test.mjs — four tests passed.
  • node scripts/build-docs-site.mjs — passed; 70 HTML pages emitted.
  • git diff --check — passed.
  • Full Swift/UI automation is not claimed for this documentation renderer fix; macOS CI remains the broader gate.

AI assistance

Prepared with Codex assistance; reproducible commands and captured production output are included above.

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
@rudycelekli
rudycelekli requested a review from a team as a code owner October 5, 2026 14:32
@clawsweeper

clawsweeper Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@cursor

cursor Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Documentation build and test-only changes; no runtime, auth, or data-handling impact.

Overview
Fixes a bug in the docs site builder where fenced code blocks without a closing ``` were silently dropped at end-of-file while the build still succeeded.

After the markdown line loop finishes, any open fence is now flushed through the same highlightCode path used for explicitly closed blocks, matching CommonMark’s EOF-as-closer behavior.

Tests: adds tests/docs-site-eof-fence.test.mjs (EOF vs explicit close, syntax highlighting, and HTML escaping inside code fences). test:docs-site now runs tests/docs-site-*.test.mjs so the new regression is included in test:safe.

Reviewed by Cursor Bugbot for commit a36fc6b. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread tests/docs-site-eof-fence.test.mjs Fixed
@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Oct 5, 2026
@clawsweeper

clawsweeper Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed October 5, 2026, 11:30 AM ET / 15:30 UTC (Revision 2).

ClawSweeper review

What this changes

The PR preserves fenced code at document EOF, adds rendering and HTML-escaping regression tests, and includes them in the docs test suite.

Merge readiness

✅ Ready for maintainer review

The fix remains necessary: current main and v4.8.0 still discard code from an unclosed fence. The patch is focused, has relevant production proof, and introduces no actionable correctness findings.

Priority: P2
Reviewed head: a36fc6b5d62a499921689eb8514d2e2bff0227bb

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused repair with relevant real-builder output, regression controls, and no remaining actionable findings.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (terminal): The captured terminal output exercises the production documentation builder with an EOF-terminated fixture and shows code emission restored after the fix; the unchanged docs tree also builds successfully. No stored-data contract changes.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The captured terminal output exercises the production documentation builder with an EOF-terminated fixture and shows code emission restored after the fix; the unchanged docs tree also builds successfully. No stored-data contract changes.
Evidence reviewed 7 items Introduced change: The pinned merge-base-to-head delta contains only the package test command, four production lines flushing an open fence, and 38 regression-test lines.
Still missing on main: Current main accumulates fence lines but finishes the parser without emitting a pending fence. The PR uses the same highlighting and attribute-escaping path as explicit fence closure.
Latest release check: The v4.8.0 builder also finishes without flushing an open fence; the requested behavior is not present in the latest supplied release.
Findings None None.
Security None None.

How this fits together

Peekaboo’s documentation builder converts repository Markdown into HTML published through GitHub Pages. Its fence parser sends captured code through syntax highlighting before emitting a code block.

flowchart TD
  A[Markdown documents] --> B[Documentation builder]
  B --> C[Collect fenced code]
  C --> D[Closing fence or document end]
  D --> E[Highlight and escape code]
  E --> F[Generated HTML pages]
  F --> G[GitHub Pages]
Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test delta production +4, tests +38; 1 test command changed The production growth directly repairs the missing EOF emission and is backed by focused rendering controls.

Technical review

Best possible solution:

Keep the EOF flush on the existing highlighting path and retain explicit-closure and literal-HTML controls.

Do we have a high-confidence way to reproduce the issue?

Yes: current-main source clearly drops a pending fence at EOF, and the contributor supplies before/after output from the actual builder with that fixture. This read-only review did not execute it.

Is this the best way to solve the issue?

Yes: flushing the captured code through the existing explicit-closure rendering path is a narrow repair that preserves highlighting and escaping.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 91cd87e79bf0.

Labels

Label changes:

No label changes.

Label justifications:

  • P2: This repairs a bounded documentation-rendering defect that silently omits code examples.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The captured terminal output exercises the production documentation builder with an EOF-terminated fixture and shows code emission restored after the fix; the unchanged docs tree also builds successfully. No stored-data contract changes.
  • proof: sufficient: Contributor real behavior proof is sufficient. The captured terminal output exercises the production documentation builder with an EOF-terminated fixture and shows code emission restored after the fix; the unchanged docs tree also builds successfully. No stored-data contract changes.

Evidence

What I checked:

  • Introduced change: The pinned merge-base-to-head delta contains only the package test command, four production lines flushing an open fence, and 38 regression-test lines. (scripts/build-docs-site.mjs:441, a36fc6b5d62a)
  • Still missing on main: Current main accumulates fence lines but finishes the parser without emitting a pending fence. The PR uses the same highlighting and attribute-escaping path as explicit fence closure. (scripts/build-docs-site.mjs:441, 91cd87e79bf0)
  • Latest release check: The v4.8.0 builder also finishes without flushing an open fence; the requested behavior is not present in the latest supplied release. (scripts/build-docs-site.mjs:441, 4d43dc9d80cd)
  • Real builder proof: The captured PR body reports the actual builder entrypoint processing an owned EOF-terminated Markdown fixture: before, zero code blocks and no copyable command; after, one code block and the command present. It also records a successful build of the unchanged shipped docs tree producing 70 HTML pages. This directly exercises the changed rendering path.
  • Security comment resolved: The head commit removes the test-only generic tag-stripping assertion flagged by CodeQL and replaces it with exact highlighted-output comparisons plus literal-HTML controls for both EOF and explicit closure. The contributor’s explanation is at fix(docs): render fenced code at the end of a document #969 (comment); no production sanitization bypass was introduced. (tests/docs-site-eof-fence.test.mjs:20, a36fc6b5d62a)
  • Feature-history routing: Current-main file history identifies repeated documentation-builder work by Peter Steinberger and metadata work by williamclay8. GitHub commit metadata verifies steipete as author and committer of the documentation-hub commit. Exact source-line introduction was not established. (scripts/build-docs-site.mjs, 81ddeb43eb57)

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • williamclay8: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)
  • reviewed 2026-10-05T14:35:45.118Z sha ad75638 :: needs maintainer review before merge. :: none

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
@steipete

steipete commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

The owner consolidation is #992, combining the five related rendering repairs after independently reproducing them on current main. Its 24 renderer tests, metadata tests, lint, full site build, and P0–P2 review pass. It also includes actual Chrome before/after screenshots using identical synthetic input. This original will be closed as superseded once the combined candidate completes CI and lands; contributor credit is retained.

@steipete

steipete commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Thanks for reporting the dropped final fence. Main reproduces it, and owner PR #992 includes the EOF flush with rendering regressions. Closing this duplicate in favor of the consolidated pending fix.

@steipete steipete closed this Oct 7, 2026
steipete added a commit that referenced this pull request Oct 7, 2026
Consolidate the reproduced CRLF metadata, entity/link, heading identity, EOF fence, and highlighter token defects from #939, #931, #940, #969, and #971. Keep highlighted fragments separate from source text and retain the shared renderer gate.

Reconcile the Unreleased note and menu-preparation documentation with current main. All 25 docs-site regressions pass; independent Codex review is clean through P2.

Co-authored-by: Rudy Mizrahi Celekli <47457359+rudycelekli@users.noreply.github.com>
steipete added a commit that referenced this pull request Oct 7, 2026
Documentation rendering lost literal code text, leaked CRLF front matter into articles, double-escaped TOC text and link queries, reused heading anchors, and discarded fenced code at EOF. This consolidates the verified fixes from #939, #931, #940, #969 and #971, with credit to @rudycelekli.

The highlighter now keeps rendered fragments separate from source text, with no reserved source characters or 6,400-token limit. The page-wide heading allocator preserves natural anchors and assigns unique duplicate/fallback IDs. Front matter is normalized before extraction; renderer-owned entities are decoded once; link suffixes remain intact; EOF flushes the pending fence. Normal macOS CI now runs the complete shared renderer gate.

Preserve the existing regression and platform proof from the PR. Reconcile with current main and retain the Unreleased changelog. Independent Codex review is clean through P2, and the final exact-head CI checks pass.

Co-authored-by: Rudy Mizrahi Celekli <47457359+rudycelekli@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants