docs(fleet-triage): design for org-wide PR/issue triage routine - #51
Open
asachs01 wants to merge 2 commits into
Open
docs(fleet-triage): design for org-wide PR/issue triage routine#51asachs01 wants to merge 2 commits into
asachs01 wants to merge 2 commits into
Conversation
Specs the automation gaps dependabot-janitor does not cover: liveness, issues, non-Dependabot PRs, the code-owner-blocked backlog, release decoupling, and agent-authored fixes. Not a second merge policy. Dependabot merging stays dependabot-janitor.sh's job; the routine consumes its output buckets rather than re-classifying. That script encodes two production incidents (sentinelone-mcp#31 vacuous-green all-skipping checks, node-datto-rmm#46 grouped-PR hidden major) that a fresh implementation would re-learn. Survey findings against main (2026-08-17) recorded in the CHANGELOG: - janitor disabled_manually since 2026-07-21; 27 days, 241 open Dependabot PRs, nothing alerting on the gap - dependabot-backlog.md 404s on main, never committed, despite a named downstream consumer - required_status_checks NULL on 20/20 sampled *-mcp repos - #23 still a live hole (grouped + green CI); #22 obsolete, would downgrade actions/checkout v6.0.3 -> v4.3.1
Ran dependabot-janitor.sh with DRY_RUN=true locally (no production state touched, disabled workflow left disabled). 100 repos scanned: 118 would merge across 71 repos, 25 majors, 21 red CI, 1 conflict. Two things the data changed: - #23 is latent, not firing. 108 of the 118 would-merge PRs (92%) are grouped and eligible only via the blanket title shortcut, but parsing every dep out of all 108 bodies found 0 cross-major and 0 unparseable bumps. The groups are update-type-scoped today. Still worth rebasing as defence in depth; demoted from re-enable blocker. - Release decoupling is promoted to a precondition. 118 merges across 71 repos with release.yml still on push:[main] is up to 71 semantic-release -> GHCR -> Azure deploys in one unbatched wave. Also records that the dry run's blocked=0 is an artifact: the script short-circuits on DRY_RUN before the approve+merge attempt, so that bucket can never populate. It neither confirms nor refutes the code-owner component. Rollout reordered accordingly.
This was referenced Aug 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design doc only — no workflow or script changes.
Why now
Survey of org automation on 2026-08-17 found two silent absence-of-signal failures running concurrently:
dependabot-janitorhas beendisabled_manuallysince 2026-07-21 — 27 days, 241 open Dependabot PRs — and nothing alerted on the gap. It produced zero failed runs because it produced zero runs.dependabot-backlog.md, whichdependabot-janitor.shwrites and whose comments name a downstream weekly digest routine as its consumer, 404s onmain. It has never been committed.Neither is reachable by hardening the merge policy, which is where all prior effort went (#28, #36, #38, #23). So liveness is component 1 of the design, not an afterthought.
What this specs
The gaps
dependabot-janitordoes not cover: liveness/deadman, issue triage (65 open), non-Dependabot PR triage (~175 open), the code-owner-blocked backlog, release decoupling, and agent-authored fixes.Explicitly not a second merge policy. Dependabot merging stays
dependabot-janitor.sh's job; the routine consumes its output buckets rather than re-classifying. That script encodes two production incidents a fresh implementation would re-learn the expensive way.Findings that need a decision regardless of this doc
required_status_checksisNULLon 20/20 sampled*-mcprepos. CI passing is enforced nowhere at the branch level. The janitor'sgh pr checksread is the only CI gate in the system, and it has been wrong twice (fix(janitor): don't auto-merge grouped/major bumps with no CI to verify them #36, fix(janitor): treat all-skipping CI checks as no-CI, not green #38) with nothing behind it.classify()retains the blanketgrep -qiE '\bgroup\b' -> ELIGIBLEtitle shortcut; fix(janitor): don't auto-merge grouped/major bumps with no CI to verify them #36 added only a downstream guard for grouped PRs with no CI. A grouped PR with green CI still auto-merges on a title match — thenode-datto-rmm#46shape minus the no-CI leg.CONFLICTING; needs a rebase onto post-fix(janitor): don't auto-merge grouped/major bumps with no CI to verify them #36/fix(janitor): treat all-skipping CI checks as no-CI, not green #38main, not a rewrite.actions/checkouttov4.3.1;mainalready SHA-pinsv6.0.3via ci: bump deprecated Node 20 actions to Node 24 versions #24. Recommend closing with a pointer to ci: bump deprecated Node 20 actions to Node 24 versions #24.MERGEABLE/CLEANand ready.Open questions in the doc
task_1786765529531, per fix(janitor): add cortextos+conduit to dependabot-janitor repo scope #50) — the janitor was turned off deliberately. Nothing should re-enable it until this is answered.blockedbucket.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.