This repository ports and re-imagines the Gemma documentation with Zensical. The immediate goal is a polished proof of concept that is safe to merge and easy to continue maintaining.
This file is a living roadmap, not a changelog. Completed implementation detail belongs in Git
history; durable authoring instructions belong in README.md, docs/contributing/, or
skills/gemma-docs/SKILL.md.
- The Zensical design system, navigation, page-context header, global Gemma 4 selector, and MathJax formula rendering work.
- Agent-facing raw Markdown routes and
llms.txtare generated by the build (docs/llms.txtis committed and verified bylint_docs.py). - Twenty-three Markdown-first tutorials export 66 Jupyter notebooks (
docs/notebooks/*.ipynb) with Apache 2.0 license headers, including model-specific variants for.model-swappages and dynamic Colab link switching in the browser. - Three interactive demos (FunctionGemma physics playground HF Space, AIventure web game, and
Embedding Draw Challenge), a task-first Learn hub, visual guides (
tutorials/visual-guides/), and the 6-part Gemma Explained series (tutorials/gemma-explained/) demonstrate richer documentation experiences. - Google Analytics 4 (
G-CWMC2KTTGQ) and cookie-consent management are configured inzensical.toml(withGA_MEASUREMENT_IDoverride support inscripts/build_docs.py) and verified bytests/test_analytics.py. - All 65 imported pages in
scripts/importers/sources.tomlare frozen (frozen = true) and hand-maintained across 97 total documentation pages. - The lint baseline (
.lintbaseline.json) is empty (0 new, 0 knownacross all 15 checks). - Repository governance (
CONTRIBUTING.md,CODEOWNERS, Apache 2.0LICENSE), contributor guides (docs/contributing/), and the authoring skill (skills/gemma-docs/SKILL.md) are complete.
Deferred demos and interactive tools are not POC merge blockers.
Keep this list short. These tasks materially affect production cutover or repository maintenance; broader product ideas belong in the deferred roadmap.
Currently, docs/notebooks/*.ipynb are committed to main and strictly validated by
scripts/lint_docs.py --check generated so Colab links (blob/main/docs/notebooks/...) work
directly across all model sizes. If removing committed .ipynb files from main is desired later:
- Publish notebooks from CI to a dedicated
generated-notebooksbranch. - Point every Colab URL (and
model-switcher.jsURL rewriter) at that branch. - Remove committed
.ipynbfiles frommainonly after the published URLs work. - Give
contents: writeonly to the post-build publishing job on successfulmainbuilds, using SHA-pinned actions and remaining zizmor-clean.
Five legacy upstream pages remain unported (/meme-generator is delinked in link_map.toml):
/core/gemma_library/core/pytorch_gemma/core/deploy_to_cloud_run_from_ai_studio/gemma_chat/meme-generator
Decide post-merge whether to permanently retire or port any of these legacy guides.
Imported content is validated upstream. A port changes formatting, not prose or code.
- Do not reword, reorder or silently correct imported prose during conversion.
- Do not alter imported code without an explicit technical reason and review.
- Every imported page records
source_urlin frontmatter. - Every existing manifest entry is frozen. Never reconvert frozen pages in bulk.
- A new page is added to
scripts/importers/sources.tomlunfrozen, imported and reviewed once, then markedfrozen = truebefore documentation-specific editing begins. - Converter improvements affect future imports only; repair existing pages deliberately by hand.
The official-doc import pipeline is:
scripts/importers/port_docs.py
-> scripts/importers/vendor_images.py # only if the source needs local assets
-> scripts/importers/rewrite_links.py
-> scripts/lint_docs.py --baseline
port_docs.py runs the Markdown converter and fidelity check before writing. Visual guides use
scripts/importers/import_newsletter.py instead. Temporary demo/vendor images are not worth
reworking because they will be replaced with original PNGs.
- Add
.notebook-cellonly to fences that should execute in Jupyter. - Ordinary prose becomes Markdown cells.
.outputfences are omitted and recreated when the notebook runs.- Site-only regions use the notebook skip markers documented in
docs/contributing/components.md. - Tutorials with
.model-swapcode blocks export separate model-specific notebooks (<stem>-gemma-4-<size>.ipynb) for all candidate models matching the page's engines andrequires:modality constraint. Non-swappable tutorials export a single<stem>.ipynb. - Frontmatter
colab:specifies the default model notebook (-gemma-4-e2b.ipynb), anddocs/javascripts/model-switcher.jsdynamically rewrites the Colab link when a reader chooses another model. - Exported notebooks automatically include Google LLC Apache 2.0 license header cells.
- Do not maintain Markdown and notebook JSON in parallel.
data/models.yml,data/frameworks.yml,data/links.yml, anddata/glossary.ymlare the single sources of truth for model metadata, engine facts, shared links, and glossary tooltips. Do not hand-maintain copies in pages or JavaScript.- Referenced Zensical icons are vendored into
overrides/.icons/viascripts/vendor_icons.pyso standalone CI lint jobs run without installing Zensical. - Frontmatter keys can shadow template data. Template macros use
modeldata.*for this reason. - Measure rendered layout in a browser. Source inspection alone is not sufficient for visual work.
- Keep Zensical pinned. A package update must not silently restyle the documentation.
- New workflow changes must use least privilege, SHA-pinned actions and pass zizmor.
The directory layout separates production infrastructure, optional import utilities and tests.
| File | Responsibility |
|---|---|
build_docs.py |
Production entry point: generators, GA4 config injection, Zensical build and raw Markdown publication. |
agent_docs.py |
Builds llms.txt and raw .md routes. |
lint_docs.py |
CI gate for documentation and repository invariants across 15 checks. |
frontmatter_schema.py |
Shared frontmatter schema and controlled vocabularies. |
generate_model_names.py |
Compiles model/framework data for templates and the browser selector. |
generate_snippets.py |
Renders data/links.yml and data/glossary.yml into snippets/links.md and snippets/glossary.md. |
notebooks.py |
Exports registered Markdown tutorials to single or multi-model .ipynb files. |
vendor_icons.py |
Vendors referenced Zensical SVG icons into overrides/.icons/. |
pinned_links.py |
Scans and reports external URLs pinned to a specific model generation (data-model). |
check_contrast.py |
Validates WCAG contrast ratios for the CSS brand color ramp (--gemma-blue-seed). |
These remain supported because more official Markdown and notebook content is expected.
| File | Responsibility |
|---|---|
convert_notebook.py |
Convert official .md.txt/notebook exports into Zensical Markdown. |
port_docs.py |
Manifest-driven, freeze-aware import orchestration. |
check_fidelity.py |
Detect prose lost during a new conversion. |
rewrite_links.py, link_map.toml |
Convert upstream links after import. |
vendor_images.py |
Optional remote-asset step for a new import. |
import_newsletter.py |
Import additional first-party visual guides; remove if that roadmap is dropped. |
sources.toml |
Import provenance and frozen-page safety boundary. |
Regression tests and converter fixtures belong in tests/, not beside executable maintenance
commands. CI discovers test_*.py from this directory (test_agent_docs.py,
test_analytics.py, test_convert_notebook.py, test_notebooks.py).
Removed from the maintained surface:
backfill_frontmatter.py: completed one-time migration.PLAN-ARCHIVE.md: duplicated Git history and made the supported workflow harder to find.
Run before handoff or merge:
uv run --frozen python -m unittest discover -s tests -p "test_*.py"
uv run --frozen python scripts/lint_docs.py --baseline
uv run --frozen python scripts/build_docs.pyFor a new official-doc import, additionally run the relevant importer in --dry-run mode and
review its fidelity report before writing files.
- Continue the Markdown-first notebook ingestion/export workflow, but keep it one-way after import.
- Add visual-guide assets only when original, production-quality PNGs are ready.
- Expand the simple task-first Learn page only when new content justifies another route.
- Keep the FunctionGemma HF Space and AIventure demo as the initial interactive examples.
- Defer additional spaces, media embeds and interactive demos until the core content is settled.
- VRAM and context calculator driven by
data/models.yml, with model, quantization, context, batch size, KV-cache dtype and GPU presets; encode state in the URL. - In-browser E2B q4 playground via WebGPU/transformers.js with an explicit large-download opt-in.
- "Ask Gemma about Gemma" documentation assistant.
- Interactive Gemma 4 architecture and MTP explainers.
- DiffusionGemma denoising animation.
- Community throughput leaderboard, only if its maintenance cost is justified.
- Self-hosting cost calculator.