Repository navigation
ci(runners): send fifteen fleet fallbacks to the larger-runner group - #26809
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design problems found. Every change is a one-line revert, the fork tail keeps forks on a runner they can reach, and the routing reuses Notes
[DESIGN-REVIEWED] b26ad1b Description read[DESCRIPTION-READ] 034468baa555979ab77886b16a48bd159eb01d9df514582c175945a808c056ac That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: this applies the existing What this change shipsInventory (6 items) — 6 justifiedIntent: when the CodeBuild fleet turns away a run in this repository, it waits in the org's own larger-runner queue instead of the enterprise-shared pool. This is an ADDITION: a CI runner move, and not a product-shape change.
[FIRST-PRINCIPLES-REVIEWED] b26ad1b Description read[DESCRIPTION-READ] 929030bfd0d53ef6f350addb134487a4642e52392b7ea392ed6f6aa953607c14 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
bolichen97
left a comment
There was a problem hiding this comment.
Approving head b26ad1b.
I read every workflow hunk. All 15 changes edit only the last fallback of a runs-on: expression: a run the CodeBuild fleet turns away now gets kirocrew-linux-4-core when the run is in this repository, and ubuntu-latest otherwise. The fleet branch, every if:, the triggers and the job steps are unchanged. That label is already used by other jobs on main, and this finishes the class started by #25482, #25645 and #26170.
Checked against the workflow safe-read rules: no secret or token use is added, no permissions: change, no new pull_request_target or workflow_run trigger, no download-and-execute, and no checkout plus run of PR code in a credentialed job. Fork pull requests still run on pull_request with a read-only token, so moving them to the larger-runner group changes only where they queue and what they cost, not what they can reach.
0 checks are red, and the Design, First Principles, Opus and GPT lanes pass. Only two backend shards and the macOS smoke job are still running, and auto-merge waits for PR Readiness.
Problem / Motivation
Goal: When CI turns a pull request away from the CodeBuild fleet, its jobs land on our own larger-runner group instead of the runner pool the whole company shares.
Fifteen jobs ask the CodeBuild fleet for a runner first — fourteen pull-request lanes, plus
ci.yml'scancel-superseded-run, whoseif:makes it a push tomain. The fleet admits only a same-repository run, from an allowlisted actor, on a narrow set of events. Every run it turns away goes toubuntu-latesttoday, the standard pool whose queue is shared enterprise-wide. #25482, #25645 and #26170 already did this for other jobs; these fifteen are what is left of the class.Why it matters
CI for these lanes waits in a queue we cannot do anything about, and it happens far more often than "forks only" suggests. The fleet admits the
pull_requestactionsopenedandsynchronizeonly, while these workflows also subscribe toreopened,edited,labeledandunlabeled— so every bot label and every description edit on an ordinary same-repository pull request misses the fleet and lands on the shared pool today.Measured over a complete 6-hour window from the runner each job actually got: 450 jobs would move. On the two workflows carrying most of them, 317 of 357 (88%) were same-repository runs on a non-admitted action, and only 40 (11%) were fork pull requests.
Not a goal
main-ratchet-audit.yml's four jobs. They stay hosted, but not for the reason the doc previously gave. The merge-capacity argument does not single them out: the queue pollsci.yml,fast-gate.ymlandbuild.ymlas well, and this diff moves their merge-group leg for any unlisted queuing actor, while severalci.ymljobs reach the group unconditionally already. The doc now says that plainly. What actually keeps the ratchet audit out is narrower — its hosted tail is pinned by a dedicated assertion rather than derived, so moving it means deletingtest_the_ratchet_audit_route_stays_off_the_larger_runner_groupon purpose, which deserves its own change and its own evidence. It also measured zero moving jobs in the same window.kirocrew-linux-4-coreis 4 vCPU and 16 GB, the same asubuntu-latest. Nothing gets faster; the change is about queue position.What changed (motivation → approach → change)
A turned-away job now waits in our own queue instead of the company's. Nothing runs faster —
kirocrew-linux-4-coreis 4 vCPU and 16 GB, the same machineubuntu-latestgives you. What changes is who is ahead of you in line: the standard pool's queue is shared enterprise-wide, and this group's is ours.Each of these fifteen jobs picks its runner with one
runs-onexpression: ask the fleet, otherwise fall back. The fallback was a single hosted image. It is now two branches, and which one a job takes depends only on where the run executes.kirodotdev/KiroCrewubuntu-latestkirocrew-linux-4-coreubuntu-latestubuntu-latest, unchangedThe third row is why this is not a one-word swap. A personal fork cannot reach an organization runner group at all, so a job there asking for that label would sit queued until GitHub cancelled it, which is worse than the shared pool. The repository test in the fallback is what keeps a fork on the image it can actually reach.
This shape already exists in the repository as
_GROUP_THEN_HOSTED_TAIL, used by the five non-pull-request sites that moved earlier. Two canonical constants now reuse it instead of ending in a bare hosted literal, so all fifteen lines stay byte-identical to one definition rather than drifting apart.docs/ci/ci-and-reviews.mdcarried the old boundary in writing: it named five fleet-fallback sites and explained that the batch was held to non-pull-request lanes precisely so it would add no pull-request-time occupancy. This diff makes that false, so the doc now names all twenty sites, says plainly what the fifteen cost and what they buy, and keeps the rollback recipe and themain-ratchet-audit.ymlexclusion intact.Capacity. This adds pull-request-time occupancy to a capped group the review lanes already compete for, which is what
docs/ci/ci-and-reviews.mdpreviously said the first batch was scoped to avoid. Over the measured window the added concurrency peaked at 34 job-slots, p95 17, median 4, and the moving jobs billed about 878 minutes in those 6 hours.Two caveats on that number, both load-bearing. The group's live
maximum_runnersis 200; this page said 100 in three places, so every capacity argument written against it was wrong by a factor of two, and the diff corrects them. And the group's current peak occupancy is not measured here — the number that decides this is current peak plus 34 against the cap. That is not the single API query it sounds like (there is no org-wide jobs endpoint; it means walking jobs for every run of every on-group workflow in a window), so it is called out rather than quietly skipped. These lanes fire on the same pull-request traffic as the lanes already there, so the peaks coincide rather than averaging out.Rollback is per line and does not disturb the fleet route: drop the middle branch so the tail reads
|| 'ubuntu-latest' }}again, and replace_GROUP_THEN_HOSTED_TAILin the constant with that literal rather than deleting it.Backwards compatibility
Compatible: no run loses a destination. A run that reached the fleet still reaches it, and a run executing outside this repository still gets the same hosted image. Only the upstream non-fleet branch changes, and it changes to a runner that exists.
Removes nothing: the diff adds a branch to an expression and widens two test expectations; no key, path, job, or capability is removed or renamed.
Tests
test_ci_fleet_routing_expression_parity.pypins every one of the fifteenruns-onstrings against the two canonical constants, so a line that drifts from the shared definition fails.test_ci_additional_fleet_routes.pyevaluates each expression across the actor, event and repository truth table and reads the expected non-fleet value off the policy itself, so it covered the move without edits.test_ci_windows_canary.py::test_all_actual_routes_require_repository_event_and_admitted_actorhad pinned the non-fleet destination forci.yml'schangesandawait-fast-gateto the single literalubuntu-latest. That expectation is now two-valued by repository. The test's own contract — fleet admission requires repository, event and an admitted actor — is untouched, and the assertion is strictly more specific than before: it now distinguishes the upstream destination from the fork one where previously it asserted one value for both.main-ratchet-audit.ymlis asserted untouched:test_the_ratchet_audit_route_stays_off_the_larger_runner_groupstill passes unmodified.Manual verification
N/A — routing is evaluated by the truth-table tests above, which is stronger than running one pull request and seeing where it landed.
Related Issues
Closes #26802
Checklist