Skip to content

feat: support addon-defined Handlebars generators#1197

Open
gennaroprota wants to merge 7 commits into
cppalliance:developfrom
gennaroprota:feat/template_only_generators
Open

feat: support addon-defined Handlebars generators#1197
gennaroprota wants to merge 7 commits into
cppalliance:developfrom
gennaroprota:feat/template_only_generators

Conversation

@gennaroprota
Copy link
Copy Markdown
Collaborator

@gennaroprota gennaroprota commented May 8, 2026

This adds support for Handlebars-based generators that don't involve any C++ code. <addon>/generator/<name>/ directories are picked up at config-resolve time and installed as additional generators; escape rules can be provided via <name>/mrdocs-generator.yml.


Adds support for addon-defined Handlebars generators: any <addon>/generator/<name>/ directory is picked up at config-resolve time and installed as a generator, with escape rules and other per-generator options described in <name>/mrdocs-generator.yml. New output formats can now be added entirely from configuration and template files, without touching the C++ generator infrastructure.

To make this work cleanly the generator subsystem is made data-driven. Escape rules previously lived in a dedicated AdocEscape subclass; they now come from data, and the file is removed. Each output format used to have its own HandlebarsGenerator / AdocGenerator / HTMLGenerator subclass; those classes are collapsed into thin wrappers over a shared data-driven implementation. The test runner was rearranged to match: generator lookup is deferred until the per-test settings load, since addon-defined generators now register dynamically rather than at static init.

A small bug fix rides along: getFileText no longer fails on empty files.

Changes

  • Source: New src/lib/Gen/hbs/AddonGenerators.{cpp,hpp} discovers and registers addon-defined generators at config-resolve time, wired into src/tool/GenerateAction.cpp. The manifest understands an optional escape mapping (per-character replacement table) and an optional displayName scalar; both fall back to sensible defaults when absent. src/lib/Gen/adoc/AdocEscape.{cpp,hpp} removed; escape rules now come from data consumed by the existing generators. AdocGenerator, HTMLGenerator, and HandlebarsGenerator reworked to read their options from data; the per-format subclass headers shrink correspondingly. src/lib/Support/Handlebars.cpp and the public include/mrdocs/Support/Handlebars.hpp gain a small surface for the new flow. src/lib/ConfigOptions.json and docs/mrdocs.schema.json updated for the new config keys. src/lib/Support/Path.cpp fix for getFileText on empty files.
  • Tests: New src/test/lib/Gen/hbs/AddonGenerators.cpp unit test covers generator discovery and registration. src/test/Support/TestLayout.{cpp,hpp} and src/test/TestRunner.{cpp,hpp} reworked to defer generator lookup until per-test settings load, so dynamically registered generators are visible to the runner. New end-to-end fixture under test-files/template-only-generators/mock-md/ ships a complete addon (addons/generator/mock-md/ with mrdocs-generator.yml and Handlebars layouts) and a simple.mock-md expected output.
  • Build: CMakeLists.txt updated to install the new sources and pick up the new test fixture (+38 lines).
  • Tooling: One line removed from util/generate-config-info.py after a config-option rename.
  • Breaking changes: None at the user-facing CLI. The generator subclasses (AdocGenerator, HTMLGenerator, HandlebarsGenerator) and the removed AdocEscape are internal — downstream code that included those headers directly will need to use the shared data-driven entry points. The ConfigOptions.json / mrdocs.schema.json schema gains keys but removes none.

Testing

  • src/test/lib/Gen/hbs/AddonGenerators.cpp is the unit-level coverage for the new discovery/registration logic.
  • test-files/template-only-generators/mock-md/ is the end-to-end fixture: a complete addon-defined generator (template-only, no C++) wired together with an mrdocs-generator.yml and a simple.cpp input that produces simple.mock-md. Any regression in discovery or in the data-driven generator pipeline fails this fixture.
  • The existing escape-rule and generator-output golden tests act as regression coverage for the data-driven rewrites: collapsing the per-format subclasses changes the code path but must not change the output, and the unchanged golden output proves that.
  • No CI workflow changes needed — the existing src/test/ and test-files/ jobs already run both the unit test and the end-to-end fixture on every build.

Documentation

  • docs/mrdocs.schema.json updated to reflect the new mrdocs-generator.yml keys.
  • New "Custom output formats" subsection under "Generator Templates" in docs/modules/ROOT/pages/generators.adoc covering the user workflow: directory layout, the *.<name>.hbs filter that gates whether a directory is picked up as a generator, the optional mrdocs-generator.yml keys, and the first-addon-wins layering rule.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 8, 2026

🧾 Changes by Scope

Scope Lines Δ% Lines Δ Lines + Lines - Files Δ Files + Files ~ Files ↔ Files -
🛠️ Source 56% 619 556 63 15 2 13 - -
🧪 Unit Tests 36% 394 371 23 11 7 4 - -
📄 Docs 5% 54 47 7 2 - 2 - -
🏗️ Build 3% 38 38 - 1 - 1 - -
🧰 Tooling <1% 1 - 1 1 - 1 - -
Total 100% 1106 1012 94 30 9 21 - -

Legend: Files + (added), Files ~ (modified), Files ↔ (renamed), Files - (removed)

🔝 Top Files

  • src/test/lib/Gen/hbs/AddonGenerators.cpp (Unit Tests): 237 lines Δ (+237 / -0)
  • src/lib/Gen/hbs/AddonGenerators.cpp (Source): 198 lines Δ (+198 / -0)
  • src/lib/Gen/hbs/HandlebarsGenerator.hpp (Source): 130 lines Δ (+129 / -1)

Generated by 🚫 dangerJS against bcb2e1f

@gennaroprota gennaroprota force-pushed the feat/template_only_generators branch from 97fc42c to 1d5651d Compare May 8, 2026 10:47
@codecov
Copy link
Copy Markdown

codecov Bot commented May 8, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.12%. Comparing base (baf81eb) to head (bcb2e1f).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #1197   +/-   ##
========================================
  Coverage    82.12%   82.12%           
========================================
  Files           33       33           
  Lines         3149     3149           
  Branches       734      734           
========================================
  Hits          2586     2586           
  Misses         387      387           
  Partials       176      176           
Flag Coverage Δ
bootstrap 82.12% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cppalliance-bot
Copy link
Copy Markdown

cppalliance-bot commented May 8, 2026

An automated preview of the documentation is available at https://1197.mrdocs.prtest2.cppalliance.org/index.html

If more commits are pushed to the pull request, the docs will rebuild at the same URL.

2026-05-29 14:24:07 UTC

@gennaroprota gennaroprota force-pushed the feat/template_only_generators branch from 1d5651d to b08c8a0 Compare May 8, 2026 14:35
@alandefreitas
Copy link
Copy Markdown
Collaborator

Add a brief note about how this change was tested (or why tests are not needed). :)

@alandefreitas
Copy link
Copy Markdown
Collaborator

alandefreitas commented May 28, 2026

Thanks. The "data-driven extension" framing is great, and the addon path is exactly the shape we were asked for.

Scope of the data-driven rework

I'm not so sure about removing the built-in AdocGenerator and HTMLGenerator and using the same path for them (their escape now goes through EscapeMap::apply instead of whatever the compiler was doing). That won't have the same performance as the built-ins, which is important because the default ones are used a lot. We should keep them on their existing fast path.

The *.<name>.hbs filter

The rule is a bit subtle: a directory is or isn't a generator depending on whether some file inside it happens to be named after it. Could we make it explicit instead, like "a directory under <addons>/generator/ is a generator only if it ships an mrdocs-generator.yml"? That gives users a clear opt-in and we get rid of the special case for common/.

Process-global registry in the test runner

The deferred lookup is the right fix. One thing to flag: discoverAddonGenerators writes into a process-global registry that's never cleared between tests, and maybeRegister skips a name that's already taken. Once a test registers mock-md, every later test gets the same one. Two fixtures can't ship competing same-named generators. Probably what you want, but worth a comment in the code so we don't get caught by it later.

displayName

I'm not sure how much this actually shows up for an addon-defined generator. The commit message says "shown in --help and similar contexts", but --help doesn't read a config file, so discoverAddonGenerators hasn't even run by then. The only place I can find displayName() being called at runtime is the "Generated X documentation in Y" line in Generator::build. Is that what you had in mind, or am I missing something?

Single-byte escape keys

The single-byte limit is too tight for something meant as a general way to add output formats. A few real cases it rules out:

  • Markdown **bold**: a single * rule can't tell emphasis tokens from a literal asterisk; an ** pattern can.
  • Escaping the escape character itself: \ to \\ works, but \\ to \\\\ for a literal pair can't be written.
  • RST inline code: (literal) and ` `` (emphasis) need separate rules.
  • UTF-8: anything past ASCII gets indexed byte by byte today, which silently does the wrong thing for escapes that should apply per codepoint.

The performance argument doesn't need to keep us stuck with single-byte keys. Keep the 256-entry array as the fast path for single-byte rules. Only check a second structure (a small vector of patterns, or a trie) when a multi-byte rule actually starts at this position. The hot path stays the same array lookup; the secondary path only runs when there's a longer pattern to match. So we get the longer patterns working without paying for it when no one uses them.

This introduces `EscapeMap`, a pattern-replacement table used to escape
rendered output values. Single-byte sources live in a 256-entry array
indexed by `unsigned char`; multi-byte sources go into a secondary array
of buckets keyed by first byte, so the walk pays nothing extra for the
multi-byte machinery in the common case (most buckets are empty). When a
bucket is non-empty, the longest match wins, so a `**` rule takes
precedence over a `*` rule at the same position. The default
`HandlebarsGenerator::escape()` walks the map. This is the path
addon-defined Handlebars generators use: each format's rules come from
data (loaded later from `mrdocs-generator.yml`), rather than being
hardcoded in a C++ subclass.

The built-in `adoc` and `html` generators keep their existing
hand-written `escape()` overrides, because the table lookup is slightly
slower than the compiled switch they had and those generators sit on the
hot path. `HandlebarsGenerator::escape()` is therefore left virtual so
the built-ins can override it.

Multi-byte support is what makes the map workable as a general escape
mechanism: it accommodates tokens like Markdown's `**` versus a literal
`*`, RST's `` `` literal `` `` versus `*emphasis*`, and UTF-8 codepoints
past ASCII that want to be replaced as a unit instead of byte by byte.

The pre-existing public `HTMLEscape` helper is refactored to read from
the same shared character-to-entity table (`htmlEscapeEntities`), so the
table can be used both there and by an `EscapeMap` for any addon-defined
HTML-like generator.
This moves `id`, `fileExtension`, and `displayName` from per-subclass
virtual overrides to base-class members set through the constructor.

That's a prerequisite for letting users add a Handlebars-based generator
without writing C++.
The function was failing on `!file.good()` which includes the `eof()`
case, so an empty file caused a failure. Test for "fail() but not just
because we hit EOF", instead.
An <addon>/generator/<name>/ directory that ships an
mrdocs-generator.yml file is now installed as an additional
`HandlebarsGenerator` at config-resolve time. The manifest's mere
presence is the explicit opt-in; its content is read for escape rules.
An empty file is valid.

This way, a generator can be added without writing any C++.
Generator lookup happened once at `TestRunner` construction. With
addon-defined Handlebars generators now possible, a generator
contributed via a test's `addons-supplemental` was unreachable from the
test binary: the lookup ran before that test's mrdocs.yml had been
loaded, so the generator wasn't yet registered.

The runner now defers the lookup. The built-in generators are not
affected.
This adds a fixture under test-files/template-only-generators/mock-md
that ships its own mock "Markdown-like" generator via an addon-local
`addons-supplemental`: a one-line layout that emits the symbol name,
plus a single escape rule mapping `_` to `\_`.
@gennaroprota gennaroprota force-pushed the feat/template_only_generators branch from bbe59b8 to bcb2e1f Compare May 29, 2026 14:13
@gennaroprota
Copy link
Copy Markdown
Collaborator Author

gennaroprota commented May 29, 2026

Thank you for the thorough review, as usual.

I addressed all of your points. Specifically:

Scope of the data-driven rework

Agreed and reverted. The built-in AdocGenerator and HTMLGenerator now keep their escape() overrides; AdocEscape.{cpp,hpp} is back. EscapeMap::apply is now the default that addon-defined generators use, and HandlebarsGenerator::escape() is virtual so the built-ins override it.

The *.<name>.hbs filter

Switched to manifest-as-discriminator. A directory under <addons>/generator/ is treated as a generator iff it contains a mrdocs-generator.yml. The file's mere presence is the opt-in; its content is read for escape rules; an empty file is valid. The common/ special case goes away.

Process-global registry in the test runner

Added comments at maybeRegister and around the deferred lookup in TestRunner.cpp flagging the persist-across-tests behavior, so future readers aren't caught.

displayName

You're right that the only runtime surface for an addon-defined generator's displayName is the "Generated X documentation in Y" info line. I dropped the the key entirely. The HandlebarsGenerator constructor still takes a displayName (the built-ins pass "Asciidoc" / "HTML"), but for addon-defined generators it's just the directory name now.

Single-byte escape keys

Implemented your sketch. EscapeMap keeps the 256-entry array as the single-byte fast path, plus a secondary array of buckets keyed by first byte for multi-byte rules. The walk hits the multi-byte bucket only when it's non-empty; longest match wins, so a ** rule beats a * rule at the same position. Unit tests cover Markdown-style ** and a UTF-8 codepoint replacement. The mock-md end-to-end fixture now also exercises a multi-byte rule (TODO -> [!]) via a doc-comment, so the path is covered through real rendering, not just through the unit walk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants