Skip to content

Frontier AI tutorials, and SkyRL moves into Reinforcement learning - #1949

Merged
cosmicbboy merged 4 commits into
niels/rl-tutorialsfrom
niels/frontier-ai-tutorials
Nov 9, 2026
Merged

cosmicbboy merged 4 commits into
niels/rl-tutorialsfrom
niels/frontier-ai-tutorials

Conversation

@cosmicbboy

Copy link
Copy Markdown
Contributor

Reorganizes the tutorials so that Reinforcement learning holds every RL tutorial and Frontier AI covers the workloads frontier AI labs run day to day. Adds five Frontier AI tutorials, each validated end to end on a Union cluster.

Stacked on #1938 (the reinforcement learning tutorials): merge that first.

SkyRL moves into Reinforcement learning

  • tutorials/frontier-ai/agentic-rl-skyrl is now tutorials/reinforcement-learning/agentic-rl-skyrl. The move is a rename-only commit, and the user-guide links are updated.
  • The RL section is now visible in both variants, because SkyRL has an open-source version. The Union-only tutorials are gated in the section's table and cards.
  • The redirects are in Redirects for the SkyRL tutorial move into Reinforcement learning unionai-docs-infra#363, and this PR moves the infra pointer to it.

New Frontier AI tutorials. The section index now opens with the three kinds of work these labs do (data factories, decisions from many small runs, continuous measurement) and a table of who does each workload.

Page Use case Result from the validated run
Refine a pretraining corpus Filters, dedup and decontamination on a CPU fan-out; LLM quality labels distilled into a cheap classifier 300k FineWeb documents to 3,454. The classifier is 66x cheaper per document than the LLM. A new threshold reruns in 33 s from cache
Evaluate every checkpoint Checkpoints published as artifacts mid-training; an OnArtifact trigger runs a cached benchmark suite on each one GSM8K 3% → 24%. A guard picks the release candidate. A backfill reruns only the new benchmarks
Curate a video dataset for generative video Scene cuts, GPU scoring and VLM captions on separate pools; the kept clips published as a dataset artifact 569 candidate clips from CC-BY Blender films, 369 kept
Find features in a language model with sparse autoencoders One cached activation harvest, 18 SAEs trained in parallel, and Claude labels scored for detection 93–98% loss recovered. Detection score 0.73 against 0.51 for mismatched labels
Closed-loop molecular discovery A surrogate picks molecules, an xTB oracle measures them on a CPU pool, and a human approval gate sits before each batch 31 hits against 4 for random search on the same oracle budget

Process

  • I chose the five use cases from research into what frontier AI labs publish and hire for, then cut and critiqued the candidates for diversity and feasibility.
  • After the first validated runs, an independent review found code/doc mismatches and cache-key gaps. All of them are fixed, and the affected tutorials were rerun.

Pointers. The unionai-examples pointer moves to the commit in unionai/unionai-examples#349, and unionai-docs-infra to the commit in unionai/unionai-docs-infra#363. Both should move to their repos' main once those PRs merge.

Checks. markdownlint, cspell, make check-links, check-images, check-subpage-cards, check-icon-names and check-deleted-pages pass, and make dist builds both variants.

🤖 Generated with Claude Code

cosmicbboy and others added 4 commits November 8, 2026 09:40
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
…oth variants

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
…ideo curation, sparse autoencoders, closed-loop discovery

The section index now frames frontier AI by the workloads labs run day to day,
with a table of who does each one and what makes it hard at scale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
…nfra to the SkyRL redirects

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

GHA build & deploy preview

Built by .github/workflows/build-pr.yml and deployed to the docs CF Pages project by .github/workflows/deploy-pr-preview.yml.

Branch alias https://pr-1949-niels-frontier-ai-tu.docs-dog.pages.dev
This commit https://9a7ce151.docs-dog.pages.dev
Commit SHA d550718e3b4adca54afcdfabbeda0f8847edace4

Updated automatically on every push.

@cosmicbboy
cosmicbboy merged commit fd08853 into niels/rl-tutorials Nov 9, 2026
17 checks passed
@cosmicbboy
cosmicbboy deleted the niels/frontier-ai-tutorials branch November 9, 2026 14:43
cosmicbboy added a commit that referenced this pull request Nov 10, 2026
…design, self-play league (#1938)

* Reinforcement learning tutorials: humanoid locomotion, Eureka reward design, self-play league

Add three tutorials to the Reinforcement learning section, each validated end to end
on a Union cluster, with report and artifact screenshots from the validated runs:

- Teach a humanoid to run: Brax PPO over MuJoCo MJX, 8,192 G1 robots on one GPU.
- Reward design with Claude and sandboxes: an Eureka-style reward search with every
  candidate trained in a unionai-sandbox session.
- Self-play league for Connect Four: a league of past snapshots, Elo, caching, and an
  app to play the champion.

Rewrite the section index with a comparison table, update the tutorials index card,
and link the new tutorials from the user guide. Point unionai-examples at the commit
that adds the examples (unionai/unionai-examples#348).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

* Reinforcement learning tutorials: match each card to its page description

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

* Frontier AI tutorials, and SkyRL moves into Reinforcement learning (#1949)

* Move the SkyRL tutorial into Reinforcement learning (rename only)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

* Reinforcement learning: list the SkyRL tutorial in the section, for both variants

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

* Frontier AI tutorials: pretraining data refinery, checkpoint evals, video curation, sparse autoencoders, closed-loop discovery

The section index now frames frontier AI by the workloads labs run day to day,
with a table of who does each one and what makes it hard at scale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

* Bump unionai-examples to the Frontier AI tutorials and unionai-docs-infra to the SkyRL redirects

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>

---------

Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

---------

Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant