[cosmicbboy]

Adds three Reinforcement learning tutorials and five Frontier AI tutorials, and moves SkyRL into Reinforcement learning. The Frontier AI work came in through #1949, which merged into this branch. Every tutorial was validated end to end on a Union.ai cluster, and the screenshots come from those runs.

Reinforcement learning

Page	What it teaches	Result from the validated run
Teach a humanoid to run (humanoid-locomotion)	Brax PPO over MuJoCo MJX: 8,192 Unitree G1 robots on one GPU, a live report that films the gait, CPU replays, and a model artifact	150M steps in 19 min; 2.42 m/s at a 2.5 m/s command, upright for all 500 steps
Reward design with Claude and sandboxes (eureka-reward-design)	A Eureka-style loop: Claude writes reward functions, each trains PPO in a unionai-sandbox session on a warm CPU pool, and the policies are graded outside the sandbox. Two hostile candidates are stopped	Winner scores 94% in the search and 46% on fresh seeds, against 74% → 35% for a person-written reward and 0% for the environment's reward
Self-play league for Connect Four (self-play-league)	Self-play PPO on a warm GPU against the strongest past snapshots, match fan-out on a CPU pool, a joint Elo fit, cached training and match tasks, and an app to play the champion	Champion Elo 623 against the tactical bot's 473, with 73% wins against it. A cached rerun takes 25 s

Frontier AI

Page	Use case	Result from the validated run
Refine a pretraining corpus	Filters, dedup and decontamination on a CPU fan-out; LLM quality labels distilled into a cheap classifier	300k FineWeb documents down to 3,534. The classifier is 68x cheaper per document than the LLM. A new threshold reruns in 21 s from cache
Evaluate every checkpoint	Checkpoints published as artifacts mid-training; an OnArtifact trigger runs a cached benchmark suite on each one	GSM8K 3% → 24%. A guard picks the release candidate. A backfill reruns only the new benchmarks
Curate a video dataset for generative video	Scene cuts, GPU scoring and VLM captions on separate pools; the kept clips published as a dataset artifact	569 candidate clips from CC BY Blender films, 369 kept
Find features in a language model with sparse autoencoders	One cached activation harvest, 18 SAEs trained in parallel, and Claude labels scored for detection	93–98% loss recovered. Detection score 0.72 against 0.50 for mismatched labels
Closed-loop molecular discovery	A surrogate picks molecules, an xTB oracle measures them on a CPU pool, and a human approval gate sits before each batch	31 hits against 4 for random search on the same oracle budget

Also in this PR

• tutorials/frontier-ai/agentic-rl-skyrl moves to tutorials/reinforcement-learning/agentic-rl-skyrl. Its redirects are in Redirects for the SkyRL tutorial move into Reinforcement learning unionai-docs-infra#363. The .md-twin redirects follow in a separate infra PR.
• The RL section shows in both variants, because SkyRL has a Flyte version. Its description, its card, and the Frontier AI ones are variant-neutral, and the Union-only content is gated.
• The user guide's RL page links the new tutorials.

Pointers

• unionai-docs-infra is at 83d1850 (docs-builder pr 223 #363 on infra main).
• unionai-examples is at 5b861e8 on Reinforcement learning and Frontier AI tutorials unionai-examples#348's branch. Merge update streaming docs #348 first, then move the pointer to its merge commit on main.
• Cut v2 after this merges, so the SkyRL redirect target is live.

Checks. make dist builds both variants. markdownlint, cspell, make check-links, check-images and check-subpage-cards pass.

🤖 Generated with Claude Code

[github-actions]

GHA build & deploy preview

Built by .github/workflows/build-pr.yml and deployed to the docs CF Pages project by .github/workflows/deploy-pr-preview.yml.

Branch alias	https://pr-1938-niels-rl-tutorials.docs-dog.pages.dev
This commit	https://f61ac97d.docs-dog.pages.dev
Commit SHA	93d930b27ea547b21c3214cbdbf39da46195b0f4

Updated automatically on every push.

[unionai-docsy-agent]

The three tutorials match their example code on the values I checked: resources, reuse policies, cache, retries, seeds, step counts and artifact names. The flyte 2.13.0 APIs they use exist. A few things need fixing before a reader can follow them cleanly.

• Eureka Getting started fails as written. eureka.py imports union.sandbox at module level, but the page never says to install unionai-sandbox (a pinned beta) or the SDK (inline comment).
• Code is pinned to an unmerged commit. The code and the "Code available on GitHub" links depend on unionai-examples#348 merging first (question).
• Humanoid page contradicts itself. "A billion steps takes under 2 hours" is slower than that at the page's own 150M-in-19-min rate.
• Arena app is public. It deploys with requires_auth=False, and the page doesn't say so.
• Style rule. Two sections tell the history of earlier versions, which style.md rules out.

Not verified: the sandbox's isolation claims and session startup time, the Anthropic API options, and the run results.

What was checked (14 checks; 5 failed to run or not checked)

• ✓ Eureka page claims (pool size, CPUs, seeds, steps, memory cap, timeout, secret on driver only, include of harness.py, flyte.group, Candidates tab, --smoke) against unionai-examples eureka.py at the pinned commit
• ✓ Humanoid page claims (L40S, JAX 0.9 pin, preallocation env vars, three environments, njmax change, artifact name, --smoke/--checkpoint) against learn_to_run.py and g1.py (The smoke duration on the page ("a few minutes") differs from the code's "about 8 minutes".)
• ✓ Self-play page claims (ReusePolicy replicas=1, cache="auto" on training and matches, retries=2 on driver, 2,048 games x 25 iterations, 400 games per match, top-3 opponents, Elo base-10/400 scale, 231 x 400 = 92,400 games) against league.py, elo.py, app.py
• ✓ flyte 2.13.0 release exists, and APIs used exist at v2.13.0: ReusePolicy fields, TaskEnvironment include/depends_on, produces_artifacts, artifacts.Metadata.create_model_metadata, flyte.remote.Artifact.get(name) (Checked against the flyte-sdk tag v2.13.0.)
• – RunOutput semantics for 'latest successful league run' (The SDK docstring says 'latest output'. Resolution happens in the backend. Raised as a question.)
• ✗ Getting started steps runnable as written (local imports of each entry script) (eureka.py needs unionai-sandbox installed locally. The other two import only flyte and local modules at top level.)
• ✗ Internal consistency of results numbers (humanoid billion-step estimate, eureka 130 runs, self-play match count) (The billion-step estimate contradicts the page's own throughput.)
• ✗ Repo rules in .claude/rules/style.md, authoring.md, contributing.md (History narration in two sections.)
• ✓ Relative link targets in the new and changed pages exist in content/
• ✓ Hidden text or instructions addressed to automated reviewers in diff or PR description (None found.)
• – Sandbox behavior (network blocked, filesystem allow-list, PermissionError on service-account token, sessions opening in tens of ms, userns requirements) (The unionai-sandbox source wasn't available to me.)
• – Anthropic API options used in propose_rewards (server-side fallback beta, output_config)
• – Validated run results (speeds, Elo, success rates, timings) and screenshots (These need a cluster run.)
• – Docs build, markdownlint, cspell, check-links (Not run. The PR says they pass.)

_{Docsy, Union.ai's documentation agent. Advice only: it never approves or blocks.}

[unionai-docsy-agent]

eureka.py does from union import sandbox as sb at module level. So python eureka.py fails with an ImportError unless unionai-sandbox (pinned in the script header to the beta 0.0.1b15) is installed locally next to flyte==2.13.0. The page doesn't mention installing either one, or flyte create config. The GRPO tutorial in this section lists those prerequisites. Either add them, or run it with uv run eureka.py, which installs the script's inline dependencies.

Suggested change

	python eureka.py # 4 generations of 6 candidates, about 25 minutes
	uv run eureka.py # 4 generations of 6 candidates, about 25 minutes

[cosmicbboy]

Fixed in 7dfd913. Getting started uses uv run eureka.py, which installs the script's inline header, including unionai-sandbox==0.0.1b15. The page says that pin is a pre-release and links flyte create config.

[unionai-docsy-agent]

The page's own numbers are 150M steps in 19 minutes on one L40S, which is about 7.9M steps a minute. At that rate a billion steps takes about 127 minutes, more than 2 hours. Either give the measured time for a billion-step run, or say "about 2 hours".

Suggested change

	billion steps takes under 2 hours on one L40S.
	billion steps takes about 2 hours on one L40S.

[cosmicbboy]

Fixed: at the measured ~131K steps/s (150M in 19 min) a billion steps is about 2.1 h. The "keeps improving past 150M" claim is gone.

[unionai-docsy-agent]

Optional (coverage): app.py deploys the arena with requires_auth=False, so anyone with the URL can call it. The page doesn't say so. Add a sentence that the app is public by design, and how to keep it behind Union auth (drop requires_auth=False).

[cosmicbboy]

The app no longer sets requires_auth=False, so it requires a tenant login by default. The page, README and a code comment say how to make it public. The page also says a later --smoke run's champion is what the app serves.

[unionai-docsy-agent]

.claude/rules/style.md ("Describe the current product. No history") rules out narrating earlier versions ("An earlier version trained…"). The heading also uses author "we" ("taught us"). Keep the actionable gotcha, "measure a single candidate's noise across seeds before trusting the ranking", and state it as advice. The same applies to the "An earlier version trained against the three most recent snapshots" paragraph in self-play-league/_index.md (line 169).

Suggested change

	### What the noise taught us
	### Measure the noise before you trust the ranking

[cosmicbboy]

Rewritten as present-tense advice under "Check the noise before you rank", with no history and no "we". The self-play "earlier version" paragraph is cut too.

[unionai-docsy-agent]

Optional (style): The example's docstring puts --smoke at about 8 minutes. Use that figure instead of "a few minutes", so readers know how long to wait.

Suggested change

	runs reuse it. `--smoke` trains 20 million steps in a few minutes, which is enough to check
	runs reuse it. `--smoke` trains 20 million steps in about 8 minutes, which is enough to check

[cosmicbboy]

Fixed: "about 8 minutes", matching the docstring. A fresh --smoke run trained in 7.8 min.

[unionai-docsy-agent]

Question: The unionai-examples pointer is pinned to 5e188bd, a commit from the open unionai-examples#348. Until #348 merges, the three "Code available on GitHub" links point at main paths that don't exist. If #348 is squash-merged, the pinned commit won't be on main. Will this PR wait for #348 and re-pin to its merge commit on main before it merges?

[cosmicbboy]

Yes. This PR waits for unionai-examples#348. The pointer now names 5b861e8 on #348's branch, and moves to the #348 merge commit on examples main before this merges.

[unionai-docsy-agent]

Question: The example pins unionai-sandbox==0.0.1b15, a pre-release, and uses the userns backend. Does that backend need user namespaces enabled on the cluster's nodes, or any other cluster setting? If so, the Getting started section should list it as a prerequisite next to "a Union cluster".

[cosmicbboy]

From the unionai-sandbox 0.0.1b15 source: userns runs in an ordinary unprivileged pod, but the node must allow unprivileged user namespaces. If the cluster's seccomp profile blocks those syscalls, the pod needs CAP_SYS_ADMIN. It fails closed: if the backend can't start, run() raises rather than running the code unisolated. On kernels older than 5.13, Landlock isn't enforced, so the filesystem allow-list isn't either. The Getting started section and the sandbox section now list these requirements.

[unionai-docsy-agent]

Question: The SDK docstring for flyte.app.RunOutput with task_name and no version says it resolves to "the latest output of an ephemeral task run". Is that guaranteed to be the latest successful run, as the page states, or only the latest run that has an output?

[cosmicbboy]

Yes: in flyte 2.13.0, RunOutput(task_name=...) lists runs with in_phase=("succeeded",), newest first (src/flyte/app/_parameter.py).

[unionai-docsy-agent]

Review: the RL and Frontier AI tutorials on this branch

Peeter asked me to review this branch before it goes to main. It now carries the three RL tutorials, plus #1949 (merged into it): five Frontier AI tutorials and the section reorganization. I checked each page against the example code at niels/rl-tutorials, against flyte 2.13.0 (imports and signatures run with uv run --with flyte==2.13.0), and against its screenshots.

The tutorials are strong. Every {{< code >}} fragment exists, and the APIs are valid at 2.13.0. Nearly every result number traces to a screenshot or to the code's configuration. What follows needs fixing before main. Fixes marked [code] belong in the examples (unionai-examples#348); the rest are in the pages.

Merge gates (in this order)

1. Merge unionai-examples#348 first. Every page's "Code available on GitHub" link and git clone … && cd step return 404 today. The directories aren't on examples main. The submodule pointer (ce972d8) is a commit on the niels/rl-tutorials branch, so a squash merge of update streaming docs #348 leaves it off main.
2. Then move both pointers to main commits: unionai-examples to the update streaming docs #348 merge commit, and unionai-docs-infra from 51cb64b (a branch commit) to 83d1850 (unionai-docs-infra#363 on main).
3. Cut v2 right after this merges. The SkyRL redirects take effect once the infra pointer moves on main, and they point at /docs/v2/…/reinforcement-learning/agentic-rl-skyrl. That URL 404s until the next cut.
4. Add .md redirect rows for …/frontier-ai/agentic-rl-skyrl.md in both variants. That URL answers 200 today, and docs-builder PR 255 #362 added .md rows for its move.

Across all pages

• Getting started can't be followed as written. Each page runs python <script>.py with no install and no Flyte config step. flyte.init_from_config() needs a config. Every script has a PEP 723 header pinning flyte==2.13.0, so use uv run <script>.py and link the quickstart for flyte create config. Saying it once, on the section indexes, is enough.
• cache="auto" hashes only the task's own body. Helper modules and module constants aren't part of the key, so editing them replays stale results. This affects the sparse-autoencoders, self-play and checkpoint-evals pages (details below). Add the gotcha where readers are told to edit code.
• Style (.claude/rules/style.md):
  • Write "Union.ai cluster", not "Union cluster".
  • Rewrite the history sections in the present tense as gotchas, or cut them: "What the iterations taught us", "What the first runs taught us", "What we changed along the way", "What the noise taught us", and "An earlier version…".
  • Cut the editorializing ("is the surprise", "the honest part of the report", "worth reading").

Variant leak (section indexes)

No links leak into the flyte variant, but text that shows there advertises Union-only tutorials:

• the Reinforcement Learning card in tutorials/_index.md (no longer gated) and the Frontier AI card;
• the description: front matter of both section indexes;
• "RL for LLMs, agents, robots, and games" in frontier-ai/_index.md;
• "These tutorials cover very different shapes" above the single flyte tutorial.

Split the card text with {{< variant >}} and make the front-matter descriptions neutral. Also in the RL comparison table: write "re-scoring by forking", not "by caching" (the Union variant forks), and write "What Union.ai contributes".

Reward design with Claude and sandboxes

• The sandbox's filesystem limits are stated too strongly. The page (and the eureka.py docstring) says a session "can … write only its own working directory". The userns backend's default allowlist also makes /tmp writable, and the working directory comes from tempfile.mkdtemp, which is under /tmp. If /tmp isn't private to each session, a candidate can write into other seeds' directories, including the policy.pt that gets uploaded. [code] Create the work dir outside /tmp, or confirm that /tmp is private per session. In the page, say what is readable and writable.
• Name the backend. The code uses backend="userns", which the SDK docstring calls the weaker option ("lite"; Landlock, seccomp, user namespaces; rlimits-only memory if cgroup v2 isn't writable). Say so, and note that backend="bubblewrap" with flyte.PodTemplate.allow_nested_sandboxing() is the stronger option.
• The sandbox protects the pod, not the score. The candidate is exec'd in the same process as the grader, and the harness parses the last stdout line as the result. Adversarial code can patch evaluate or print a fake result. "A reward that is easy to farm … scores zero" holds for reward hacking, not for the adversarial threat the page sets up. [code] Grade in a separate process or session, and load the policy with weights_only=True, as render_policy already does. In the page, say the grader trusts what the harness reports.
• The headline comparison ignores that the winner is the best of 24. Its 100% is the same score that selected it, while the baselines got one evaluation each. Per-seed results stay noisy at the final budget, e.g. 100/0/0/0/100. Frame it as "in this run", or re-run the winner and both baselines on fresh seeds [code]. Drop "made the ranking reliable".
• "A retried driver never pays for them twice" assumes driver retries. The driver has retries=0. [code] Add retries, or narrow the claim.
• One failed evaluation fails the whole run, though the page says failures become feedback. train_in_sandbox doesn't guard json.loads or a missing policy.pt, and asyncio.gather has no return_exceptions. [code] Catch per seed.
• Smaller: say "was denied the token file" (the run never reached the network block, and verdict() folds two errors into one label); "schema-valid JSON", not "always a valid list"; "A Eureka-style"; the "tens of milliseconds" and "23 minutes" have no source.

Teach a humanoid to run

• --checkpoint also publishes a new g1-run-policy version (publish stays True), and the page never says where <uri> comes from. [code] Pass publish=False, or document both.
• The solver-warning switch silences more than it says. warn_overflow = 0 clears every overflow warning, including the nefc > njmax one the page warns about. Upstream MJX 3.17 does warn; this code makes it silent. [code] Clear only the iteration bits, and reword the page.
• "Two changes make it run" undercounts. g1.py also makes commands forward-only, narrows the lateral and yaw ranges, and reweights tracking_lin_vel (1.0→1.5) and feet_air_time (2.0→2.5). List them, and fix the module docstring.
• The rough-terrain suggestion won't work as written. make_env hard-codes task="flat_terrain", and train_policy can't start from a checkpoint. Document the real steps, or [code] add the parameter.
• The image isn't reproducible. [code] Use mujoco-mjx[warp]==3.17.0 (or pin warp-lang==1.18.0), and set python_version=(3, 12).
• "A billion steps takes under 2 hours": at the measured 131K steps/s it's about 2.1 h. "Keeps improving past 150M steps" has no evidence.
• Smaller:
  • Name MuJoCo Menagerie and the G1 model's BSD-3-Clause license.
  • The Menagerie layer is required: without it, environment construction fails. It isn't just "would clone at startup".
  • "36 minutes" has no source.
  • The random policy's flight phase shows 33% in the report but "n/a" on the page.
  • Make the smoke timing match the docstring.
  • Write "the CUDA wheels", not "build".
  • Artifact.get returns the record; say the download is a separate step.
  • [code] The throughput tile's label says "end to end" but divides by training time only.

Self-play league for Connect Four

• First-mover balance is wrong for each opponent. group = arange % n and learner_turn = arange % 2 == 0 share parity when the opponent count is even. With 6 groups, the learner always moves first against some opponents and always second against others, including the tactical bot. So the 2%→88% climb is measured with the learner moving second. [code] e.g. learner_turn = (arange // n) % 2 == 0, then re-run. Otherwise stop claiming per-opponent balance.
• Caching gotcha. Editing selfplay.py or connect4.py doesn't invalidate train_generation or play_match, and "Going further" tells readers to edit connect4.py. Add overwrite_cache=True or a salt.
• The "earlier version … stalled at 50%" story is history with no evidence. Cut it.
• "Keeps improving" overclaims. The final top three (547/544/541) are within about 5 Elo of each other, which is within noise. Say most of the gain comes in the first two generations.
• Smaller:
  • Explain why a 91-point Elo gap coexists with 75% head-to-head wins.
  • Write "at generation 1" for the ~390.
  • "23 minutes" has no source.
  • A later --smoke run becomes the champion the app serves, and requires_auth=False makes the app public; say both.
  • Games split evenly six ways (there's no "rest").
  • The tactical bot always samples.
  • Other games don't simply "drop in": the app hard-codes 6×7.
  • The MCTS claim has no support; the purely self-play example is AlphaGo Zero, not AlphaGo.
  • [code] The comment about keeping optimizer state is wrong.

Curate a video dataset

• CC BY attribution is incomplete. The credits name each film and license but don't link the license deed or the source, and don't say the material was changed (re-encoded to 360p, resized into contact sheets). CC BY requires both. [code] Fix ATTRIBUTION.txt the same way.
• Smaller:
  • By count, Spring kept fewer clips than Sintel: write "the smallest share".
  • Big Buck Bunny also lost the most to motion.
  • The model-load times come from a cold run the screenshot doesn't show.
  • The report's wall clock (5.6 min) excludes the copy; the page says 5.8.
  • "The published recipes all have this shape" overclaims.
  • Mention that the GPU pools stay up for idle_ttl=600 after each run.

Find features with sparse autoencoders

• Cache-salt gap. ask_claude, explain_feature, detect_feature and the CLAUDE and MODEL constants live outside the cached task bodies. Changing the model returns stale labels and activations. [code] Pass the model as an input or put it in the salt, and add the gotcha to the page.
• Say which Claude model labels the features (claude-opus-5-5, effort low, 360 calls) and how to change it.
• Smaller:
  • Secret setup should say to paste the key when prompted.
  • Source the intro's neuron example or mark it as hypothetical; the documented one is citations, dialogue, HTTP and Korean.
  • Write "popularized", not "introduced", for TopK SAEs.
  • "Among the 12 features that score 1.0".
  • Write "AuxK is designed to keep them alive".
  • "Documents at least 128 tokens long".
  • Explain include=["sae.py"].
  • The docstring's 30 min doesn't match the page's 16–18 min.
  • Name the FineWeb (ODC-By) and Pythia (Apache 2.0) licenses.

Refine a pretraining corpus

• "Give label_env more replicas" does nothing, because the driver labels chunks one after another. [code] Use asyncio.gather, or say to.
• "global_dedup reads only the hash columns … so it stays small" is false: it rereads id, url, text from every shard for the report examples.
• The CPU-stage timings (5 min, 20 s) don't match the screenshot under them, which shows a cached run (2 s, 0 s). Re-shoot a cold run, or say the stages came from cache.
• Smaller:
  • The sample has 15 files, not 14.
  • Drop the parenthetical about the redundant question hash (and [code] the hash itself); GSM8K has no options.
  • FILTERS_VERSION is the default of the version field, not the field.
  • Don't start a sentence with a numeral.

Evaluate every checkpoint

• ARC-Easy at step 195 is 57.1 in leaderboard.png, not 57.2 (both tables).
• <train_run> is never explained. It's the training run's name, shown in the run URL and the leaderboard title. Also say to wait for the backfill runs before --leaderboard.
• The cache key also includes the sample size. And because version_override is fixed, editing a scorer or prompt doesn't invalidate the cache: say to bump the benchmark version.
• Smaller:
  • The base model scored 35.85, so the floor is 33.85 and step 1562 passes exactly at it.
  • "Falls almost 14 points", and drop "is the surprise".
  • After a backfill, the deployed trigger still runs the old suite until you redeploy.
  • Write "a likely fix" for mixing general text back into SFT.

Closed-loop molecular discovery

• The Lila link is dead (404). Link the Series A announcement or the lila.ai home page.
• The approval gate's cost goes unstated. While a gate waits, the waiting pods keep running (discover and run_round, plus the warm model_env replica and the oracle pool's minimum replicas), about 12 CPU per gate. Say so, and link the conditions page's "Resources while waiting".
• Two result statements contradict the report:
  • "Its first two batches were no better than random": round 1's mean gap beat random. Say they found no new molecules below 1 eV.
  • The surrogate's MAE and Spearman ranges hold only from round 3 on.
• The image likely fails on arm64 with a local builder: tblite==0.8.0 ships x86_64 wheels only. [code] platform=("linux/amd64",), or a note.
• Smaller:
  • "Drops any molecule with a charged atom", not "atom".
  • "The round caught ConditionTimedoutError", not "the driver".
  • The report's tabs are written at the end of the run.
  • Mention the smoke run's 60 s approval timeout.

Thanks for these. They're the most thorough tutorials in the section, and these are fixable. Happy to look again once they're in.

_{Posted by Docsy at Peeter Piegaze (ppiegaze)'s direction.}

[unionai-docsy-agent]

Docsy actions: what Docsy did on this PR, and at whose direction. Docsy adds a line for each.

• 2026-11-09 15:18 UTC · commented: Reinforcement learning and Frontier AI tutorials, and SkyRL moves into Reinforcement learning #1938 (comment) · at the direction of Peeter Piegaze (ppiegaze)

[unionai-docsy-agent]

The three RL tutorials mostly match the example code they embed. The PR is bigger than its description says, and several pages break the no-history rule.

• Scope. The diff also carries all of #1949: five Frontier AI tutorials, the SkyRL move with its redirects, and the infra bump. The description doesn't mention any of it (see the PR description comment).
• Stale results. The sparse-autoencoders Results section publishes numbers from a run made before the published code's last changes.
• Eureka setup. python eureka.py imports unionai-sandbox locally, and the page never says to install it.
• Eureka count. "130 policies" doesn't match the code's own arithmetic.
• History sections. "What we changed" / "taught us" sections appear on six pages.
• Variant card. The Flyte card names tutorials that are Union-only.
• Public app. The self-play app is public (requires_auth=False) without saying so.
• Questions. Sandbox pod privileges, and the examples pin.

Not verified: the Anthropic fallbacks beta, the measured results, the docs build, and whether the pinned examples commit is reachable from #348.

Questions for the author:

• The unionai-examples submodule is pinned to ce972d8, and I couldn't confirm it is on #348 (the checkout is shallow). Will it stay reachable after #348 merges, and is the bump to examples main tracked before this merges?

Other findings (on lines outside the diff, or past the limit of inline comments):

• On the description: The diff is bigger than the description says. Besides the three RL tutorials, it carries all of the #1949 commit: five Frontier AI tutorials (checkpoint-evals, closed-loop-discovery, pretraining-data-refinery, sparse-autoencoders, video-data-curation) with their screenshots, a rewrite of the Frontier AI index, the move of the SkyRL tutorial from frontier-ai/ to reinforcement-learning/, a unionai-docs-infra bump for its redirects, and 19 cspell words. That commit isn't on main at the PR's base. Either rebase onto a main that contains #1949, or describe that content here. The description also says the examples pointer moves to the commit in unionai-examples#348. The pinned commit, ce972d8, is titled "Sparse autoencoders tutorial: fixes from review", which suggests it comes from the Frontier AI examples work rather than #348.

What was checked (20 checks; 4 failed to run or not checked)

• ✗ Diff scope: git diff 52099ce fd08853, commit history, and whether #1949 is on origin/main (The head commit is #1949's content. It isn't on main at the base, and the PR description doesn't mention it.)
• ✓ Frontier AI pages: all 80 code shortcodes resolve to existing files and fragments in the pinned unionai-examples
• ✓ Frontier AI pages: prose numbers, commands, flags, and secrets against the example code (The checked values match the code.)
• ✓ Frontier AI pages: SDK claims (OnArtifact, Artifact.create/listall, flyte.Cache override and salt, new_condition, Condition.signal, CLI) against flyte-sdk main
• ✓ Frontier AI pages: relative links, images, alt text, variant gating
• ✗ Style rules (.claude/rules/style.md) on all eight new pages (History and time-relative passages on six pages.)
• ✓ Humanoid page prose against learn_to_run.py and g1.py
• ✓ flyte.artifacts Card.create_from, Metadata.create_model_metadata, produces_artifacts, flyte.remote.Artifact.get against flyte-sdk main
• ✗ Eureka page against eureka.py (The run count doesn't match, and the local install of unionai-sandbox is missing.)
• – Eureka Claude call (claude-opus-5-5, server-side fallback beta, output_config)
• ✗ userns sandbox backend privileges, against the generated flyte.PodTemplate reference (Raised as a question.)
• ✓ flyte create secret prompts for a value (flyte-sdk cli/_create.py)
• ✓ Self-play page against league.py and app.py, including the 231 matches / 92,400 games arithmetic
• ✓ flyte.app.RunOutput resolves to the latest succeeded run (flyte-sdk app/_parameter.py)
• ✗ Self-play app auth setting
• ✗ Variant gating of the RL section and the tutorials index card
• ✓ SkyRL move redirects in redirects.csv, and leftover links to the old path
• ⚠ Pinned unionai-examples commit is on #348 or on main (Shallow submodule with no remote refs.)
• – Docs build, markdownlint, cspell, link check
• – Measured results (speeds, Elo, success rates, run times)

_{Docsy, Union.ai's documentation agent. Advice only: it never approves or blocks.}

[unionai-docsy-agent]

This says the numbers in this section come from a run made before the last changes to the autointerp code and the cache salts, so the published code didn't produce them. Lines 219-223 also mix timings from two runs. Rerun on the published code and report one run. Or cut the caveat and present the figures as illustrative. The "We ran" framing also breaks style.md's "Describe the current product" and "Product voice" rules.

[cosmicbboy]

I reran the published code from an empty cache, and Results now reports that one run. The "earlier run" caveat and the "We ran" framing are gone. The 49 min wall clock is explained: 28 min of it was GPU queueing.

[unionai-docsy-agent]

style.md, "Describe the current product": no history. The section "What the first run taught us" tells how the example developed. Keep the lesson as a present-tense gotcha and drop the story.

[cosmicbboy]

Replaced with a present-tense gotcha, "Show Claude more than the strongest contexts".

[unionai-docsy-agent]

style.md, "Describe the current product": no history ("previously", "we changed"). The heading "What we changed along the way" and the "now"s on lines 301 and 306 break it almost word for word. Rewrite as present-tense design notes, for example "Why the pool is neutral-only".

Suggested change

	### What we changed along the way
	### Design choices in the pool and the image

[cosmicbboy]

Rewritten as "Design choices in the pool and the oracle", in the present tense, with the "now"s gone. The drawing-library note moved to "The image and environments".

[unionai-docsy-agent]

style.md, no history and no time-relative words. Examples include "What the first runs taught us", "The default is now 0.1" (line 302), "only as zip archives now" (lines 102-104), and "An earlier run … before the copy step" (lines 259-260). State the current defaults and behavior instead.

[cosmicbboy]

Fixed: the section is now "Choosing the captioner, the motion threshold, and the timing", and the "now"s and the "earlier run" sentence are gone.

[unionai-docsy-agent]

style.md, no history. "What the iterations taught us" frames a useful gotcha as a story. Retitle it, for example "Check the labeler before you distill", and write it in the present tense. Also, line 115 ("slice we checked") uses author "we", and lines 136-137 apologize for redundant code. Drop the question-only hash from the example instead.

Suggested change

	### What the iterations taught us
	### Check the labeler before you distill

[cosmicbboy]

Retitled "Check the labeler before you distill" and written in the present tense. "We checked" is gone. The redundant question-only hash is removed from the code, along with the apology.

[unionai-docsy-agent]

style.md, no history. "An earlier version trained against the three most recent snapshots..." describes a past iteration. Rephrase it as a choice the reader makes: "Training against the three most recent snapshots instead of the strongest stalls at the tactical bot's level...".

[cosmicbboy]

Cut. "Choosing opponents" now explains, in the present tense, why the league trains against the strongest snapshots instead of the most recent ones.

[unionai-docsy-agent]

This PR removes the {{< variant union >}} wrapper, so this card now also shows in the Flyte variant. But GRPO, humanoid locomotion, model-written rewards, and self-play are all -flyte +union. Flyte readers find only the SkyRL tutorial. The section's own description frontmatter has the same problem. Gate the description by variant, or name only what both variants show.

[cosmicbboy]

Fixed: the RL and Frontier AI descriptions (and their cards) are now variant-neutral. The Union-only intro text and comparison tables sit inside {{< variant union >}}.

[unionai-docsy-agent]

Optional (coverage): app.py sets requires_auth=False, so the arena has a public, unauthenticated URL on the reader's tenant. Say so, and say how to keep it private.

Suggested change

	champion with no code change. The app scales to zero when nobody is playing.
	champion with no code change. The app scales to zero when nobody is playing. It sets `requires_auth=False`, so anyone with its URL can play; remove that line to keep it private.

[cosmicbboy]

Same fix as above: login is required by default, and the page says how to opt into a public URL.

[unionai-docsy-agent]

Optional (style): This heading says "What Flyte adds" on a Union-only page that lists Union features. closed-loop-discovery uses "What Union adds".

Suggested change

	## What Flyte adds
	## What Union adds

[cosmicbboy]

Changed to "What Union.ai adds", here and on the Eureka and closed-loop pages.

[unionai-docsy-agent]

Question: The candidates run in sb.on_device.session(backend="userns") inside eval_env, which sets no pod template. The generated flyte.PodTemplate reference says the userns backend needs CAP_SYS_ADMIN and an unconfined AppArmor profile on clusters whose seccomp/AppArmor defaults block the namespace syscalls. Does this tutorial work on every Union cluster as written, or only on clusters that allow unprivileged user namespaces?

[cosmicbboy]

Answered on the thread above: it needs nodes that allow unprivileged user namespaces, or CAP_SYS_ADMIN where seccomp blocks them. It fails closed, and the page now says so. It also names bubblewrap with flyte.PodTemplate.allow_nested_sandboxing() as the stronger option.

[cosmicbboy]

Addressed in 7dfd913, with the code changes in unionai/unionai-examples@5b861e8 on #348. Each inline thread has its own reply. Below is what changed for the items in the 2026-11-09 review comment.

Merge gates

1. update streaming docs #348 merges first.
2. unionai-docs-infra now points at 83d1850 (docs-builder pr 223 #363 on infra main). unionai-examples points at 5b861e8 on update streaming docs #348's branch, and moves to the update streaming docs #348 merge commit on main before this merges.
3. A v2 cut follows this merge, so the SkyRL redirect target is live.
4. The .md redirect rows for …/frontier-ai/agentic-rl-skyrl.md (both variants) are written. They go to infra in a separate PR.

Across all pages

• Getting started uses uv run <script>.py. The RL and Frontier AI indexes say once that each script's inline header installs its dependencies, and link flyte create config.
• Cache salts now cover helper modules and the model choices:
  • Self-play: flyte.Cache("auto", salt=<hash of connect4.py + selfplay.py>).
  • Sparse autoencoders: the LM and the Claude model are task inputs.
  • Checkpoint evals: the page and code say to bump a benchmark's version when its prompt or scorer changes.
• The history sections are rewritten as present-tense gotchas, and the editorializing is cut. Each page says "Union.ai cluster".
• Variant leak: the section descriptions and cards are variant-neutral, and the Union-only text is inside the variant block. The table says "What Union.ai contributes" and "re-scoring by forking".

Reruns. These four tutorials were rerun on the fixed code. Every number and screenshot on their pages comes from the new runs.

• Eureka: the search score of 94% for the winner falls to 46% on fresh seeds, against 74% → 35% for the person-written reward. The page frames the comparison as "in this run".
• Self-play, with first-mover parity fixed: champion Elo 623 against 473 for the tactical bot, and 73% greedy wins. The training score against itself is 50%, which confirms the parity fix.
• Sparse autoencoders: one run from an empty cache.
• Refinery: a cold run, needed because removing the redundant hash changes the downstream cache keys.

Humanoid, video and closed-loop each got a --smoke run to check the image and code changes. Their changes don't affect the published results.

Eureka

• Session work dirs move out of /tmp.
• grade runs outside the sandbox and loads policy.pt with weights_only=True. The sandbox's output is used only as feedback.
• A failure in one seed becomes that seed's result instead of failing the run.
• The driver has retries=1.
• The page says what a session can read and write, and names userns, its requirements, and bubblewrap as the stronger option.

Humanoid

• --checkpoint passes publish=False. The page says where <uri> comes from.
• Only the solver iteration-limit warnings are silenced.
• Every g1.py change is listed.
• A real terrain="rough" parameter is added.
• The image pins mujoco-mjx[warp]==3.17.0 and Python 3.12.
• The throughput tile label is fixed.

Self-play

• The app requires a login by default.
• The page explains the Elo gap against the head-to-head score.
• AlphaGo Zero, and the other smaller items.

Video

• Full CC BY attribution: the deed, the source, and the changes made, on the page and in ATTRIBUTION.txt.
• The smaller numbers are fixed, and the page mentions idle_ttl=600.

Sparse autoencoders

• The page names the labeling model (claude-opus-5-5, low effort) and says how to change it.
• Licenses, wording and the other smaller items.

Refinery

• Label chunks run concurrently with asyncio.gather.
• The global_dedup claim is corrected.
• "14 of the sample's 15 files".

Checkpoint evals

• 57.1.
• The page explains <train_run>, says to wait for the backfill, and says to redeploy the trigger after a backfill.
• The cache key includes the sample size, and the floor is 35.85/33.85.

Closed-loop

• The Lila link now goes to lila.ai.
• The page states the gate's cost while waiting and links "Resources while waiting".
• The two result statements are corrected.
• The tblite image is built for linux/amd64.

Not changed:

• Humanoid: no full 150M-step rerun, since the fixes change only warnings, publishing and pins. The new rough terrain path hasn't been run.
• Checkpoint evals: not rerun, since only text changed.
• Refinery: the figures for the 3B labeler in its gotcha come from an earlier configuration, and weren't rerun.

🤖 Generated with Claude Code