Repository navigation
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the CodeQL and license-gate jobs now wait for the rate-limit reset where they used to fail, and the new read pool falls back to Notes
[DESIGN-REVIEWED] bb14576 Description read[DESCRIPTION-READ] aca55a303e4597d1ec34a669a92f0d0f547d36c95f330621d01b82b21bd7ab79 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The rate-limit wait for CodeQL and the license gate is backed by #39367's measurements. The new read-token pool for PR Readiness is weaker: it claims a harm nobody measured, and it does nothing until a maintainer creates a secret. Notes
What this change shipsInventory (8 items) — 7 justifiedIntent: A CodeQL or license-gate job that meets a spent shared API rate limit waits for the reset instead of failing, and readiness can read from its own pool. FIX, with one addition.
[FIRST-PRINCIPLES-REVIEWED] 7083cb8 Description read[DESCRIPTION-READ] 12a49cdf594c6e5b2be735ef1e53b1e97912d1d986fab20ed3e615b2f85892de That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review —
|
CodeQL and the Dependency License Gate make their API calls inside a third-party action, with the job token that shares one hourly REST pool with every workflow. When the pool is spent the action fails and the job is red for a reason that says nothing about the diff. The shared gh-rate-limit action now also installs gh-rate-limit-wait, the same bounded wait as a command. CodeQL uploads its results in a separate step right after a wait, once per job. The license gate waits before each review attempt, and a license the retry itself reported stays a license verdict. Neither job can pass without its action having run. Part of #39367
PR Readiness and the readiness sweep read the lanes they aggregate through the pool every lane spends. With the repository secret PR_READINESS_READ_TOKEN set, both make their reads with that token and log which pool they used and what is left of it. Unset, refused or nearly spent, they read with GITHUB_TOKEN as before. Writes, every collaborator-permission read (the evaluators included) and the sweep reserve stay on GITHUB_TOKEN: a write made with another token starts workflow runs of its own, and a token refused the permission read would drop a block or a disposition record. Part of #39367
7083cb8 to
bb14576
Compare
Problem / Motivation
Goal: A CodeQL or Dependency License Gate job that meets a spent API rate limit waits for the reset instead of going red, and PR Readiness can read from a token pool the review lanes do not spend.
Every workflow here shares one pool of 15,000 REST requests an hour through
GITHUB_TOKEN. It runs dry most hours. Issue #39367 measured it over 14 days: the rate limit is behind 255 of 256 red CodeQL jobs and 93 of 95 red Dependency License Gate jobs, about 51% of all red jobs. Those jobs fail instead of waiting, and the author re-runs or pushes again.Where the two jobs die (job logs, 2026-11-19):
Analyze (actions)github/codeql-action/analyze, atUploading results, after the scan finished##[error]API rate limit exceeded for installation.actions/dependency-review-action, twice, then our probeDependency review is not supported on this repository(both attempts), thencould not run ... (gh: API rate limit exceeded for installationBoth failing calls are inside a third-party action. No
ghwrapper reaches them. In the license gate the action hides the limit behind the words it uses for a dependency-graph outage.Why it matters
A red CodeQL or license job says nothing about the diff, but it costs a re-run (a CodeQL Python scan is about 22 minutes) and each re-run spends more of the same pool. PR Readiness reads the lanes it aggregates from that pool too, so a drained pool also delays the verdict.
Not a goal
What changed (motivation → approach → change)
Jobs die on a spent pool inside a third-party action → the existing shared helper can only wrap
gh→ extend that one helper with a wait a step can run before an action, and use it in the two jobs.1. The shared helper (
.github/actions/gh-rate-limit, added for #28477). It now also installsgh-rate-limit-wait [floor]. Same script, same job-widemax-waitbudget as theghwrapper; no second helper./rate_limitreading (not counted against the quota)coreremaining >= floorready, no waitwaited::error::names the rate limit and says it is CI infrastructure, not the code under testreadywith a warning; the guarded action then runs and decides by itselfIt retries nothing and reports nothing for the action it guards. The wrapper's own behaviour is unchanged except that its budget error now also says it is CI infrastructure.
2. CodeQL (
codeql.yml). The upload comes after a scan of up to 22 minutes, so a wait before the scan would not help.analyzenow runs withupload: failure-only, and the upload is its ownupload-sarifstep (same pinned commit, same category) right after a wait for a pool under 100 requests.There is one upload per job and no retry. At the pinned commit
upload-lib.tsvalidateUniqueCategorysets a per-category marker before it sends the request and refuses a second upload of that category in the same job, so a retry step could only fail. A failed upload fails the job, as it did whenanalyzeuploaded. The job is green only when the upload succeeded. Timeout 60 → 75 minutes, to hold the 30-minute wait budget.3. Dependency License Gate (
dependency-review.yml). Waits before the first review. The pause before the retry is the wait for the reset when the pool is spent, and the fixed 60 s otherwise. After two failed reviews the verdict step decides in this order:invalid-license-changesoutput)All three fail the job. Timeout 10 → 40 minutes.
4. PR Readiness and the sweep: a pool for reads (second commit). New repository secret
PR_READINESS_READ_TOKEN. A stepSelect the token pool for reads, identical in both workflows, asks that token for/rate_limitand logs the pool and its remaining count:readiness reads: token pool -- shared (GITHUB_TOKEN); PR_READINESS_READ_TOKEN is not setGITHUB_TOKEN, as todayreadiness reads: token pool -- readiness (PR_READINESS_READ_TOKEN), N/M remaining::warning::with the readingGITHUB_TOKENSo an expired token degrades to today's behaviour and cannot stop the required check. The token is read only through
${{ secrets.* }}into a step'senv; only counts are printed.What moves to the new pool: every
gh_retrycall inpr-readiness.yml(all are reads), and in the sweep the scan, the comment and status reads and the active-dispatch listing.pr-readiness.ymlalso ends with apr-readiness: readiness pool rate limit -- N/M remainingline.What stays on
GITHUB_TOKEN, on purpose:labeled, the license gate among them.pr_status.py --disposition-gate/--supersession-gate) whole_review_contract.author_write_verdictreads a refused permission lookup as "not a writer" and drops the record, so a token refused that read could publish a green the PR has not earned. The evaluator steps are not given the new token at all.RATE_RESERVE_PERCENT20,DISPATCH_COST25)Maintainer steps for the secret
Nothing changes until these are done.
gh secret set PR_READINESS_READ_TOKEN -R kirodotdev/KiroCrewand paste the token at the prompt (Actions secret, repository level).PR Readinessrun and findreadiness reads: token pool -- readiness (PR_READINESS_READ_TOKEN), N/M remaining.GITHUB_TOKEN.What the token must be able to read, over REST and GraphQL, on this public repository: pull requests, reviews, issue comments, labels, commit statuses, check runs, workflow runs and file contents. It needs no write access and no access to collaborator permissions: no read that depends on them is made with it.
Limit to know: such a token has its owner's pool of 5,000 requests an hour, not 15,000. When it runs under 50 the run falls back to the shared pool. A GitHub App installation token would carry 15,000, but it expires hourly and cannot be stored as a secret; it needs a mint step this PR does not add.
How the 7-day count will be taken
Same method as the issue: job conclusion plus the log line. Seven days after merge, for
codeql.ymlanddependency-review.yml:gh run list -R kirodotdev/KiroCrew --workflow <file> --status failure --created '>=<merge time>' --limit 1000 --json databaseIdconclusion == "failure"(gh run view <id> --json jobs);gh api repos/kirodotdev/KiroCrew/actions/jobs/<job id>/logs | grep -ci 'rate limit'; a job counts when the result is above 0. Both the old line (API rate limit exceeded) and the new ones (GitHub API rate limit is spent) match.The two counts are compared with 255 (CodeQL) and 93 (license gate). Those baselines cover 14 days, so the like-for-like 7-day figures are about 128 and 47. The result goes on #39367, with the number of green jobs whose log carries
waiting ...s for the reset(the jobs the wait saved).Known limits and what is left
gh-rate-limit-waitreads only the REST (core) bucket.Retry-Aftergh's error text, not headers. A secondary limit (403 or 429) waits a fixed 60 s, as before.ghwrapper (EVENT_WRAPPEDintest/test_scheduled_gh_rate_limit.py). Calls made insideclaude-code-actionare not covered.GITHUB_TOKEN.Backwards compatibility
Compatible: with the secret unset both readiness workflows run as today, and the two gated jobs change only when the pool is under 100 requests or a review attempt failed.
Removes nothing: no check, trigger, permission or verdict is removed;
analyzeno longer uploads, and theupload-sarifstep after it uploads the same file under the same category.Tests
test/test_gh_rate_limit_wait.py(new, 23 tests). Runs the action's script against a stubgh, a fixed clock and a recordingsleep; nothing sleeps, nothing reaches the network.waited; quota left →ready, no sleep; under and at the floor; reset past the budget → fails naming the rate limit and CI infrastructure; still spent after the reset; unreadable pool; one budget shared with the wrapper.continue-on-erroror anif, so no path ends green without the upload.test/test_readiness_read_token.py(new, 23 tests). Runs the selection step andgh_retryagainst a stubghthat records the token each call carried: secret set → reads use it and the write does not; unset → today's token; refused, 49 left, 0 left → fallback with a warning; the token is never printed; the opt-out read; the evaluator steps are not given the token, and each one, run with the readiness pool selected and the token in its environment, still asks for a collaborator permission with the default token; the step and function are identical in both workflows; no write goes throughgh_retry; the sweep's dispatch, republish and reserve read are not routed to the new token; the reserve values are unchanged.test/test_scheduled_gh_rate_limit.py: both jobs joinEVENT_WRAPPED(install after checkout, timeout above the wait budget).test/test_dependency_license_gate_outage.py,test/pr_readiness_sweep_helpers.py: follow the added quota read and the added step.Proof: PROVEN test/test_gh_rate_limit_wait.py::test_a_spent_pool_is_waited_out_once_until_its_reset, test/test_gh_rate_limit_wait.py::test_a_pool_with_quota_left_is_not_waited_for, test/test_gh_rate_limit_wait.py::test_a_pool_under_the_floor_is_waited_out, test/test_gh_rate_limit_wait.py::test_a_pool_at_the_floor_is_not_waited_for, test/test_gh_rate_limit_wait.py::test_a_reset_past_the_budget_fails_and_names_the_rate_limit (head bb14576)
Run locally: the two new files plus the 76 existing test files that reference the touched workflows, 4,442 passed, including
test_flake_pattern_ratchet.pyandtest_gnu_only_shell_usage.py.Manual verification
Not yet verified on a runner:
Analyzerun on this PR shows whetherupload-sarifafteranalyzewithupload: failure-onlyproduces the sameCodeQLresult check. The first run was cancelled during the scan, before the upload.LICENSE_FINDINGexpression in the license gate reads the action'sinvalid-license-changesoutput withfromJSON. Its shape (forbidden,unresolved) is taken from the action's source at the pinned commit, not from a run.waiting ...s for the resetin a job log the first time one meets it.sharedline.Seen on a runner: the Dependency License Gate passed on the first push of this PR with the helper installed and the wait step in place (run 44463165421).
No workflow linter (actionlint, zizmor) is wired in this repository or installed locally; the five YAML files were parsed with PyYAML and their step scripts run by the tests above.
Real-machine verification
N/A -- CI workflow only
Related Issues
Part of #39367. Related: #36090, #32110 (not changed here). Earlier fixes this builds on: #28477, #35349.
Sibling sweep
Swept: git grep -nE "uses: (github/codeql-action|actions/dependency-review-action|actions/github-script|peter-evans|actions/labeler|dorny)" -- .github/workflows
Sibling: .github/workflows/codeql.yml -- changed here
Sibling: .github/workflows/dependency-review.yml -- changed here
Sibling: .github/workflows/ci.yml:145 -- not affected -- dorny/paths-filter reads the diff from local git, pinned by test/test_paths_filter_local_git.py
Sibling: .github/workflows/build.yml:390 -- not affected -- dorny/paths-filter reads the diff from local git, same pin
Sibling: .github/workflows/macos-on-demand.yml:131 -- not affected -- dorny/paths-filter reads the diff from local git, same pin
Sibling: .github/workflows/claude-review.yml -- tracked in #39367
Pattern harvest
Rule candidate: review-prompt
Pattern: a job whose third-party action calls the GitHub API with the job token, with no rate-limit wait before it and no fail-closed handling after it
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commitsContribution License Agreement
Contributed under the project's Apache-2.0 license.
AI-assisted, opened by Claude Code