Skip to content

ci: a release PR runs a fast release check instead of the heavy suites - #6524

Merged
miguel-heygen merged 5 commits into
mainfrom
ci/release-pr-fast-checks
Oct 27, 2026
Merged

miguel-heygen merged 5 commits into
mainfrom
ci/release-pr-fast-checks

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What changed

A release PR (versions, release notes, changelog) ran the same heavy CI as a code change, because every packages/*/package.json it bumps matches packages/**. scripts/ci/release-only-diff.mjs now reads the diff against the PR's current base and says release-only only when every changed file is a manifest (package.json, plugin.json, gemini-extension.json) whose only change is version, releases/v*.md or docs/changelog.mdx. A manifest that changes anything else, or any other file, makes it false, so a release PR that carries code, or is merely named release/, keeps the full suite. Main, the merge queue and scheduled runs never take the shortcut.

When true, the existing code, cli, cli_bundle, skills, catalog_index, gcp_beginframe, studio and edit_accuracy outputs in ci.yml, the changes outputs of regression, Windows render, preview-regression and player-perf, and new changes jobs in player-safari and lambda-arm64-image read false (skipped checks pass, as for a docs-only PR). Kept: Build (packs the manifests, checks the packed-manifest budgets, keeps the release tarballs), Test reachability, the title and Codex plugin checks, and a new Release checks job running release-only-diff.mjs --verify: every bumped manifest moves to one version, releases/v<version>.md exists with no TODO or style note, and the changelog has the entry.

What I measured

PR #6510 (0.8.207), runner minutes by workflow: CI 126, regression 92, Windows render 52, Safari perf 19, player perf 15, Lambda image 14, CodeQL 10, Docs 3, preview 2.5 (about 340 before the merge, critical path about 19 min: CI, Windows and Safari each finish at 19). A main push at the same base ran about 113 runner minutes (CI 65, Windows 41 when cancelled). After this change a release PR keeps Build 4.7, change detection about 2, reachability 0.5, the release check and the small checks: about 20 runner minutes, about 6 minutes of wall time. CodeQL and Docs still run.

  • scripts/ci/release-only-diff.test.mjs, 8 tests on the devbox: a real-git release diff is release-only and consistent; the same release plus one code file is not; a manifest with a changed dependency is not; a branch named release/ that carries code is judged by its files; an empty diff and an ordinary code PR are not; --verify refuses mixed versions, missing notes, an unreviewed TODO and a missing changelog entry; the workflow wiring is asserted.
  • fails without the fix: with the other.length === 0 condition removed the test file fails (4 of 8 fail).
  • Run against the real merged 0.8.207 release commit: release only, 21 files, --verify passes. Against the commits before it: not release only.
  • merge-queue-workflows, publish-workflow and pr-edit-concurrency tests pass with the workflow edits.

What I did NOT exercise

I did not run the workflows on GitHub: the first real check is the next release PR (its run should show the heavy jobs skipped and Release checks green). check-edit-accuracy-filter.test.mjs needs the full package tree and was not run; the filter itself is unchanged. Which jobs are required checks I did not read from branch protection; the skipped jobs behave as they do for a docs-only PR.

No visible change

CI configuration only.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 170 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (2)

  • seqplainkeys-none-center-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
  • seqplainkeys-none-pct-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 2 important · 0 nits

No blocker: the merge queue still runs the full suite (merge_group → release_only=false), so this only trims the PR stage, and every required check is either kept or skips the way it does for a docs-only PR. The two important items are inline. All required checks pass on 23e9ec72. Fallow audit (not required) is red on classify, which this PR adds.

Scope

Read the whole diff. Ran release-only-diff.test.mjs (8/8 pass). Ran the script against the 0.8.207 release 619b8bc4d: release only, --verify exits 0. Against 0390f3f90 (docs): not release only. Against a diff that only touches the changelog: release_only=true, --verify exits 1. Checked the required contexts in rules/branches/main: Lambda and Safari aren't in the list and only run on path triggers. Every heavy job checks !cancelled() or result != 'success', so a failed detection still runs it.

}
other.push(path);
}
return { releaseOnly: paths.length > 0 && other.length === 0, other, versions };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

important: if a PR touches only docs/changelog.mdx or a releases/*.md file, with no manifest bumped, this returns releaseOnly: true. That covers a weekly changelog:weekly --write digest, a changelog typo fix and an edit to old release notes. Release checks then runs --verify and fails with manifests move to different versions: none changed (I reproduced exit 1). It isn't a required check, but those PRs get a red check they can't fix. Requiring at least one bumped manifest fixes it:

Suggested change
return { releaseOnly: paths.length > 0 && other.length === 0, other, versions };
return { releaseOnly: versions.size > 0 && other.length === 0, other, versions };

The empty-diff case is still false. Add a test that a changelog-only diff is not release-only.

Comment thread scripts/ci/release-only-diff.mjs Resolved

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 1 resolved, 1 open, 0 new · 23e9ec7 → 68c22f7. Lambda arm64 image (not required) failed twice: a curl TLS error while downloading ffmpeg sources, unrelated to this change.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit d94132e Oct 27, 2026
105 of 111 checks passed
@miguel-heygen
miguel-heygen deleted the ci/release-pr-fast-checks branch October 27, 2026 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant