Skip to content

AGENTS.md

Latest commit

 

History

History
134 lines (104 loc) · 19.6 KB

AGENTS.md

File metadata and controls

134 lines (104 loc) · 19.6 KB
Raw

Agent context — documentation-generator (docgen)

Session contract (non-negotiable)

Cloud agent bc-fd13dd23-…2ccd ran ~22 hours (2026-09-07 → 2026-09-08), opened PRs #73–#149, and spawned recursive “hunt remaining bugs” children. That loop is forbidden. Details: .cursor/rules/no-factory-loop.mdc.

  • Finish the user's list in this run. Do not stop after the first item to ask for a merge, rebase, review, or permission to continue.
  • Do not ask the user to merge. Keep working on the same branch and the same PR. They merge when they want.
  • One request → one branch → one PR unless they explicitly asked for separate PRs. Update that PR each turn; do not open a new PR per file or per YAML key.
  • Batch isomorphic holes (typed config keys, or default coerces, the same fail-closed helper on the next field) into that one PR — or stop and list leftovers. Never one-key-per-PR.
  • When the list is done, STOP. Do not hunt the next milestone, spawn “hunt remaining / fail-closed holes” agents, or invent milestones/<one-config-key>.md as the next task.
  • If the next patch is the same shape as the last (new milestone file, same helper, different key): hard stop. Report. Do not open another PR.

North star

Stable goals for this repository:

  1. Embeddable generator — pip install docgen (git URL or editable install from this repo), a consumer bundle (docgen.yaml + hints/narration), and shell/CI are enough to build and maintain narrated demos. Do not vendor this library into a product repo’s src/; pin via requirements-docgen.txt / pipx / uv tool. No IDE assistant is required; optional docgen wizard is a local web app only.
  2. Hybrid config and prose — docgen.yaml should stay maintainable: deterministic merges (yaml-generate, gap checks) plus optional LLM (OpenAI or Grok) where it adds value (narration hints, declarative scene YAML). Prefer Git-reviewed changes over opaque single-shot generation.
  3. Video stack — Long-form demos pair Markdown narration, TTS (OpenAI or xAI), Whisper-style timestamps, Manim visuals, compose (ffmpeg), concat, and validate (sync and narration lint). The CLI also supports pages for static preview sites.
  4. Stable contracts — CLI, exit codes, and reusable workflows should stay predictable for downstream repos and automation.
  5. Library, not app — There is no in-repo dogfood bundle. Consumer projects (e.g. course-builder) are the integration test of record. The library must not import or special-case any consumer.
  6. Tool-only generation — Narration, merged docgen.yaml, compiled scenes.py, TTS audio, composed media, and other generated artifacts must come from docgen (CLI/library) and committed wrapper scripts that call it — not from hand-edited outputs passed off as sources. In a consumer bundle, prefer hints/*.md + yaml-generate over ad-hoc YAML surgery. Cursor rules: .cursor/rules/docgen-tools-only.mdc, .cursor/rules/no-asset-edits.mdc.

Protected assets in a consumer bundle (Cursor must not edit)

docgen (OpenAI or Grok) is the only path that produces category C outputs — see .cursor/rules/no-asset-edits.mdc. Summary (paths relative to a consumer bundle, typically docs/demos/):

  • Outputs (do not hand-edit): <bundle>/docgen.yaml (as emitted by yaml-generate); <bundle>/narration/*.md (except README.md); <bundle>/animations/scenes.py, timing.json, animations/specs/*.scene.yaml (scene pipeline); <bundle>/audio/*.mp3; <bundle>/images/*.png (scene image assets from image-generate); <bundle>/recordings/** where applicable.
  • Inputs (maintainer-owned): <bundle>/hints/** with YAML front matter (docgen.segment, docgen.wiring); maintainer scripts under the bundle; tests/** fixtures inside this library; <bundle>/narration/README.md.

docgen avoids hardcoding consumer segment ids in library code; tests may use concrete fixtures.

Consumer resets (generic)

Downstream repos that pin this library should:

  1. Run docgen yaml-generate (and review the diff).
  2. Regenerate narration, scenes, audio, and video with the documented CLI sequence for their bundle.
  3. Run validate / validate --pre-push before pushing.

Pin bumps (pip install …@<sha>) are routine when adopting a new docgen commit.

What this repo is

docgen is a documentation and narrated demo video toolkit: CLI + library focused on Manim (diagram-heavy segments), TTS, timestamps, composition, validation, pages, and wizard-assisted authoring. It is not the product application and not the CI orchestrator for downstream apps.

The Playwright/VHS/demo-function/per-function/discover-tests/catalog surface area was removed; see the README for what is supported today.

CLI surface (today)

Commands registered on the docgen CLI include:

  • ai-status — print the resolved AI provider and which env var supplied the key (never the secret).
  • init — scaffold bundle layout and docgen.yaml.
  • wizard — local web UI for narration/bootstrap workflows (focus files, in-place narration revise, per-segment asset freshness + rebuild-from-here, Vue Benchmark view, Tool tab to pip-upgrade docgen and pin requirements-docgen.txt).
  • gui — desktop window over the same Vue/Flask UI (pip install 'docgen[gui]' for pywebview). --smoke is a headless HTTP check. PyInstaller spec: packaging/docgen-gui.spec. Frozen apps resolve templates/static/benchmark JSON via docgen.resources.
  • freeze — docgen freeze builds the docgen-gui onedir (pip install 'docgen[packaging]'). Optional --smoke runs the binary headless. Do not run a full freeze in routine pytest; set DOCGEN_FREEZE_SMOKE=1 for the optional test.
  • tts — text-to-speech for segment files (OpenAI or xAI /v1/tts).
  • timestamps — word/segment timing (timing.json). Default engine local aligns the known narration text against the mp3 offline (ffmpeg silencedetect, no API); --engine whisper uses OpenAI whisper-1 or xAI /v1/stt when ai.provider is grok. Both emit the same Whisper-shaped blocks. Failed ffmpeg silencedetect raises AlignmentError (empty stderr is not treated as full-span speech). OpenAI whisper-1 word/segment start/end must be finite JSON numbers (bool/NaN raise AIError). Grok /v1/stt rejects empty word tokens and inverted end < start intervals (AIError). Empty segments.all raises TimestampError (same as TTS) and does not leave a stale timing.json as success.
  • image-generate — render scene-spec image elements (image: + prompt: boxes) via OpenAI Images or xAI Imagine into the bundle (also runs for missing assets inside generate-all). Default align_with_docs wraps each prompt with narration + source snippets and fails when the authored prompt shares no documented terms. After the PNG is written, OCR plus a vision review (align_review, default true) check the pixels against the same corpus and retry once on FAIL.
  • manim — render Manim scenes declared in config.
  • compose — mux narration audio with visual sources via ffmpeg. With no segment ids, uses segments.all (same as generate-all), not segments.default. A type: mixed row raises ComposeError if any listed source is missing (no silent subset mux).
  • validate / validate --pre-push — drift, narration lint, Manim hints, timing_sync, story_end (last paced reveal vs audio end; hard fail), scene_assets (pre-render: stuck-board cadence, frame-budget overlaps, MANIM_FONT consistency, stale helpers / stale compiled class including hand-edited generated-region labels and run_time — hard fail; also a generate-all gate before Manim), av_sync (hard fail on --pre-push / generate-all; prefers scene-spec labels as OCR anchors), subject_beat_coverage (declarative specs vs narration topic beats; hard fail when enabled), and related visual-sync checks (ocr_scan, layout, freeze_ratio — hard fail on --pre-push / generate-all). scene_assets also fails when an image-element prompt shares no documented terms with narration/source, or when OCR (and optional vision, validation.image_asset_alignment.review) finds invented on-image terms. Missing tesseract fails ocr_scan / av_sync / layout (not skip-PASS). Missing audio or an LFS pointer fails timing_sync and recording media gates (stream_presence, av_drift, ocr_scan, av_sync) (not skip-PASS). Missing *.scene.yaml fails story_end / subject_beat_coverage for type: manim (not skip-PASS).
  • lint — narration lint helper.
  • narration-generate — LLM-assisted narration from hints and repo context; optional --revise --revision-notes for in-place edits (same contract as the wizard Revise button).
  • scene-spec-generate — LLM emits declarative *.scene.yaml; enforces frame budget + subject-beat coverage (dwell OK; cover topic shifts; reject invented labels) and image-prompt alignment (image prompt: must use documented terms from narration/source).
  • scene-compile — compile specs into scenes.py (generated regions only).
  • yaml-generate — merge defaults and hint wiring into docgen.yaml.
  • clean-bundle — remove regenerable outputs per policy.
  • concat — stitch segment videos.
  • pages — emit static HTML for demo assets.
  • generate-all — orchestrated pipeline: TTS → timestamps → scene specs (auto scene-spec-generate when animations/specs/ is empty; otherwise offline retime) → images → Manim → compose → validate → concat → pages. --regen-scene-specs forces LLM rewrite; --skip-scene-retime keeps legacy hand scenes.py only.
  • rebuild-after-audio — same as generate-all with TTS skipped (still retimes scenes after timestamps).
  • benchmark — score the packaged scene-timing corpus (no bundle / Manim / OpenAI). Executes compiled construct() on the real _TimedScene clock and diffs src/docgen/benchmark_data/baseline.json. Exit 1 on regression. --update-baseline only after an intentional improvement.

Implications for changes here

  • Manim / scenes.py (marker blocks): Fix generators under src/docgen/** (manim_scene_support.py, scene_spec.py, scene_spec_generate.py, validate, yaml_generate, tests). Do not patch generated classes inside a consumer's animations/scenes.py between BEGIN/END GENERATED SCENE markers; re-run scene-spec-generate / scene-compile --retime and manim instead. Preferred consumer order: narration → TTS → timestamps → scene-spec/compile → Manim → compose. The unit CI job runs pytest tests/, which includes this module on a temp fixture: a hand-edited generated-region label or run_time fails scene_assets.
  • Beat sync (fail-closed): when timing.json has words, every story box label must match a spoken phrase (wait_word); unmatched labels and leftover LLM indices are rejected. Opt out with pace: none. Legacy row-level wait_segment is upgraded to wait_word and written back on scene-compile (direct compile_scene_class still rejects leftover wait_segment). Fuzzy containment matching is not used. scene-compile clamps FadeIn / page-fade run_time against the next word start so _TimedScene._clock cannot race past waits (issue #66 — do not emit cascading first-board dumps). After a reveal, a dwell slot may play Indicate / Circumscribe when the gap to the next wait_word is long enough (also clamped). Long holds emit additional mid-hold pulses (timed_wait + emphasis) so the board does not freeze after the first Indicate. Optional box fields: shape (rounded/pill/diamond), reveal (fade/grow/slide), emphasis (none/pulse/ring). Page transitions (fade, slide, rise, zoom, none) play the previous page's exit and the next page's first node in one timed_play (crossfade, slide, rise, or zoom) so the canvas does not empty between them. none still removes instantly. They FadeOut those boxes individually, not the parent VGroup. Durations stay clamped like other reveals. scene-compile refreshes stale _box / _arrow / _TimedScene helpers in scenes.py. A page may set chart (kind: bar or line) instead of rows. Each series item is its own paced reveal: a bar grows from the left, or a line point fades in and the segment from the previous point is drawn. The value's digits must appear in the narration.
  • Subject-beat coverage: implemented in scene_spec.layout_density_violations / cluster_subject_beats; enforced by scene-spec-generate and validate (validation.subject_beat_coverage.enabled, default true). Not a blind label count.
  • Image-prompt alignment: implemented in image_align.image_prompt_alignment_violations; scene-spec-generate and image-generate reject prompts that share no documented terms; validate reports the same failures on scene_assets (validation.image_prompt_alignment.enabled, default true). image-generate also wraps the Images API prompt with narration/source (image_generation.align_with_docs, default true).
  • Image-asset alignment: OCR + vision review of the generated PNG (docgen.image_align). image-generate fails closed and retries once (image_generation.align_review, default true). validate folds the same pixel gate into scene_assets (validation.image_asset_alignment, OCR on by default, vision opt-in).
  • docgen benchmark is required after clock / compile / _TimedScene / dwell changes. Pytest string assertions are not a substitute. Do not remove the CI benchmark job. See Required gate below.
  • Prefer stable CLI / library contracts and documented exit codes so CI can depend on them.
  • narration_from_source: hints in config + docgen narration-generate — owner-supplied context paths, not opaque bulk edits to outputs.
  • Avoid duplicating long orchestration docs here; link to downstream repos when describing their publish pipelines.

Required gate: docgen benchmark

Clock, compile, scene-spec motion, _TimedScene helpers, and dwell/hold changes must stay green on the packaged corpus:

  1. Run docgen benchmark (or ./scripts/benchmark-scenes.sh). Exit 0 vs src/docgen/benchmark_data/baseline.json.
  2. Keep tests/test_scene_benchmark.py in the default pytest tests/ run.
  3. Keep the benchmark job in .github/workflows/ci.yml (it must invoke docgen benchmark). Do not delete or skip it.
  4. --update-baseline only when the scorecard change is the point of the PR; review the JSON diff. Do not bump the baseline to hide a regression.
  5. When production fails in a new way, add a case to standard_cases() and update the baseline in the same PR.

String assertions on compiled scenes.py and simulate_reveal_timeline are not a substitute — the harness executes construct() on the real clock.

Testing (downstream relevance)

Tests should cover CLI-visible behavior and contracts that adopters rely on: yaml-generate, scene-spec-generate, scene-compile, validate, compose, generate-all, pages, init, benchmark, gui / freeze (--smoke only in default pytest), freeze-safe docgen.resources paths, config loading (repo_root, env_file), and package exports. Use small in-tree fixtures; this library does not ship a dogfood bundle. Clock / compile changes must keep docgen benchmark at or above benchmark_data/baseline.json — string assertions on scenes.py are not enough. Do not run a full PyInstaller freeze in routine pytest.

Cursor Cloud specific instructions

  • Session contract first: see the top of this file and .cursor/rules/no-factory-loop.mdc. Finish the assigned list on one PR. Do not ask to merge. Do not start a YAML-key factory or recursive hunt. milestones/README.md Active is human-assigned only; an empty Active line means stop, not “pick the next config key”.
  • Virtualenv: the project is installed editable into /workspace/.venv (created by the startup update script). Shells do not auto-activate it — run . /workspace/.venv/bin/activate (or prefix the venv path) before docgen, pytest, or ruff. The docgen console script lives at /workspace/.venv/bin/docgen.
  • System deps are pre-baked in the VM snapshot (not the update script): ffmpeg + tesseract-ocr (validation/compose/OCR), plus build-essential, python3-dev, libcairo2-dev, libpango1.0-dev, pkg-config (needed to build the manim extra's manimpango/pycairo wheels). If a fresh VM ever lacks these, reinstall via apt before pip install.
  • Standard commands are in README.md / pyproject.toml / .github/workflows/ci.yml: lint ruff check src/ tests/; tests pytest tests/ -v --tb=short; required docgen benchmark (CI job benchmark); required docgen validate on a scratch init bundle (CI job validate, scripts/ci-validate-scratch-bundle.sh). The CI unit job also exports PYTHONPATH=src (not needed locally because of the editable install, but harmless).
  • OpenAI / Grok / Anthropic vs offline commands: tts, timestamps --engine whisper, image-generate, narration-generate, scene-spec-generate, and yaml-generate --llm call a provider. Keys: CURSOR_API_KEY (Cursor Cloud), then OPENAI_API_KEY (local Cursor, Claude Code, CI). Only ANTHROPIC_API_KEY → Claude chat (no TTS/images). Grok: DOCGEN_AI_PROVIDER=grok + XAI_API_KEY. docgen ai-status prints the resolved key env. Image model is image_generation.model / --model. Integration tests auto-skip without credentials. Fully offline: init, scene-compile, manim, compose, validate, lint, pages, concat, yaml-generate (no --llm), timestamps (default local engine), and benchmark.
  • Generate against another repo (do not vendor): this environment already has docgen on PATH. Point at a consumer checkout or clone URL — nothing is copied into that project's src/: docgen --repo /path/to/consumer generate-all docgen --repo github.com/org/consumer init --defaults --repo finds docs/demos/docgen.yaml. Add the consumer as a Cloud repository dependency if you clone by GitHub URL (uses GITHUB_TOKEN). Networked stages: CURSOR_API_KEY (Cloud), OPENAI_API_KEY (local / Claude Code), ANTHROPIC_API_KEY (chat), or XAI_API_KEY (Grok).
  • scene-compile gotcha: paced specs (wait_word) need a timing.json entry for that stem (docgen timestamps after TTS). Prefer scene-compile --retime after fresh timestamps; for a fully offline smoke render, author rows without wait indices only if you accept unpaced reveals.
  • No in-repo dogfood bundle: exercise the pipeline against a scratch bundle (docgen init /tmp/<name> --defaults in a throwaway git dir). Do not hand-edit consumer generated assets (see .cursor/rules/no-asset-edits.mdc).
  • Wizard / desktop GUI: docgen wizard --port 8501 is the Flask UI (bundle optional for the Benchmark view). docgen gui / docgen benchmark --gui open the Vue benchmark view in a desktop window when pywebview is installed. docgen gui --smoke is the headless HTTP check (default pytest). docgen freeze builds the GUI onedir; do not run a full PyInstaller freeze in routine CI/pytest. Keep packaging/docgen-gui.spec and docgen.gui.packaging in sync.