Skip to content

Propose simplifications of each week's merges as draft pull requests - #1902

Merged
alistair3149 merged 2 commits into
masterfrom
ci/weekly-simplify
Nov 19, 2026
Merged

alistair3149 merged 2 commits into
masterfrom
ci/weekly-simplify

Conversation

@JeroenDeDauw

@JeroenDeDauw JeroenDeDauw commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Every Saturday at 02:00 UTC, Claude Code reads the past week's merges into master and proposes at most two behaviour-preserving simplifications of the code they changed, each as a draft pull request. It targets what per-PR review misses: duplication and leftovers that only show across PRs. While a proposal awaits review, the workflow proposes nothing new. It files no issues and posts no comments. A manual run is a dry run: it only uploads the patches. This is a smaller alternative to #1810; a table below sets the two side by side.

Human input

Goal and shape: goal set by @JeroenDeDauw in one ask; the requirements drawn from it and the design are Claude's, and @JeroenDeDauw then reshaped the bar a proposal has to clear.

  • Directions: "a from scratch implementation to get most of the value using a simple approach that is secure and doesnt create new issues or at least not ones that waste our limited attention", finished locally for his review before any PR.
  • Redirections:
    • No line-count floor on what a proposal is worth. Claude had proposed "saves ten lines or more"; @JeroenDeDauw: "sometimes removing fewer lines is still a fantastic simplification". The bar now asks what the change removes.
    • No cap on a proposal's size. Claude had capped patches at 50 added and 300 changed lines; @JeroenDeDauw: "what if the simplification can remove more? or it requires changing more lines? perhaps here we could remove the limit and see if we end up with problematic prs?" A patch now only has to remove more lines than it adds.
  • Questions that changed or tested a call:
    • "are all of those good?", of the bar's limits. It led to the one-theme rule and to the agent no longer looking for tests to delete.
    • Whether "behaviour-preserving" wrongly rules out cleaning up nonsensical edge-case handling. It stays ruled out: "general conservatism in this automating being good".
  • Choices: one theme per proposal.

Outcome: product: @JeroenDeDauw questioned the bar from Claude's summary of it and of the dry runs. Architecture and code: no human review of the files is on record. The workflow has never run in Actions. Left for you: .github/workflows/simplify.yml, the prompt in .github/workflows/simplify.md, and whether the bar suits the team.

Claude's calls

  • Not discussed with @JeroenDeDauw:
    • The requirements Claude drew from his ask, in the brief that started this work: at most two draft PRs a week, each removing more lines than it adds; no issues, reports or comments, so a bug the agent notices goes at most to the run's log; no shell or web access for the agent, a read-only job for it, and the write token in a separate job that executes nothing the agent wrote; one workflow file and one prompt.
    • The agent is Claude Code's CLI in restricted mode (pinned to 2.1.311, Opus 5.5 at xhigh), not claude-code-action. Running tests would take a shell, and a shell lets injected text send the Claude token out. So the agent runs nothing: CI on the draft PR is the only check, and a proposal can come up red.
    • It reads full commit messages, to judge what is deliberate. They are untrusted text; what contains them is the patch checks, not the prompt.
    • Up to two proposals a run, and nothing new while a simplify/ PR by ProWikiExperts is open.
    • A patch may touch only PHP, TypeScript, Vue and LESS files under four directories.
    • The agent does not look for tests to delete: nothing can check that a deleted test was redundant.
    • Saturday 02:00 UTC, outside working hours for the Claude subscription it shares. A manual run never publishes.
    • The CI workflows get permissions: contents: read in this PR, rather than through the repository setting.
  • Considered by Claude, omitted:
    • what the table shows Propose simplifications of the past week's merges every Saturday #1810 has and this leaves out: a reviewing agent and a record of earlier proposals each add a moving part, and the pause while a proposal is open prevents repeats; a report is what the ask rules out;
    • committed tests for the shell steps: they would roughly double the change, so the steps were checked locally (see the evidence);
    • retrying a failed or skipped week;
    • proposing from a fork, whose pull requests get neither a write token nor secrets in CI: it needs a fork and a token that reaches both repositories.
  • Not verified: what only shows in Actions. That is installing Claude Code on the runner, handing the patches from one job to the next, and pushing and opening pull requests with the PAT. The publish step ran only against a local bare remote with a stubbed gh.
  • Where your judgment helps:
    • Is CI alone enough of a check? Nothing here proves that a changed test still fails when the code breaks, which Propose simplifications of the past week's merges every Saturday #1810 checks before it opens a pull request.
    • Does the bar suit the team? Each run's log lists the candidates it dropped and why, which shows what the bar costs.
    • With no cap on size, two kinds of proposal would be a problem: a rewrite that barely shrinks the code, and a mass deletion. If either shows up, a cap is two lines to add back.
    • Every dry-run proposal touched only tests. Is that the kind of proposal you want?

Compared with #1810

This PR #1810
Agent's tools Read, Grep, Glob, Edit and Write in one directory; no shell, no network any shell command, with network access
Check of a proposal CI on the draft PR the agents run the covering tests, break the code once to see a test fail, and review each patch in a fresh context
Agent runs a week one three (find, implement, review), with a subagent per area
Output at most 2 draft PRs at most 3 draft PRs, and a report issue with possible bugs and rejected ideas
Earlier runs not recorded; nothing new while one of its PRs is open a history of what was proposed, merged and closed
Limits on a patch fewer lines added than removed; no cap on size no more lines added than removed, at most 600 changed
Size 279 lines; no scripts, no committed tests 1,275 lines; publish and history scripts with their test suites

The same in both: the two secrets, Saturday 02:00 UTC, Opus 5.5 at xhigh, a publish job that runs nothing the agent wrote, and read-only tokens for the five CI workflows.

The change

Start with the prompt, .github/workflows/simplify.md (85 lines): it decides what reaches your review queue. Then read the last step of .github/workflows/simplify.yml, which is the security boundary.

The repository is public, so what the agent reads and writes is untrusted. It gets only file tools, confined to a directory with the merges and the code, and nothing it writes runs in the workflow. A separate job holding the machine account's PAT checks each patch and pushes it: only PHP, TypeScript, Vue and LESS files under src/, tests/phpunit/ and resources/ext.neowiki/{src,tests}/, no mode change, symlink or binary, and fewer lines added than removed. CI then runs the proposals, so the CI workflows now get a read-only GITHUB_TOKEN instead of the repository default, which can write nearly everything. None of them needs more, and none uses a secret.

Before the first scheduled run

  • The machine account, ProWikiExperts, needs write access to this repository; it has read access. The workflow uses the CLAUDE_CODE_OAUTH_TOKEN and PR_TOKEN_PROWIKIEXPERTS repository secrets.
  • To try it once merged: run the workflow manually and read the proposals artifact. A manual run opens nothing.
  • Optional: set the repository's default workflow token to read-only, so that a workflow added later without a permissions block cannot write either.
Verification evidence
  • actionlint 1.7.12 with shellcheck is clean on simplify.yml.
  • CI on this PR ran all five CI workflows with the read-only token: 28 checks pass.
  • Publish step: its exact script ran 33 proposals against a local bare remote with a stubbed gh.
    • Rejected, each with a one-line reason: edits to .github/workflows/, to a .json, .js or docs/ file and to .gitattributes; a mode change, a symlink, a rename and binaries; a .. path, a quoted path and a path with a newline; as many or more lines added than removed; a missing title.
    • Accepted: a 301-line deletion, a new file, a deleted file, a path with a space, two proposals in one run, and a valid proposal after a rejected one.
    • @ and [skip ci] in the agent's text were neutralised, and $(…) and backticks in it never ran.
  • Collect step: an emptied tracked file became a deletion, an emptied new file vanished, and a new file with content stayed.
  • Restricted mode, probed headless with Claude Code 2.1.311: reads and writes outside the run directory, a read of /proc/self/environ and writes to .git files were refused, and only Read, Grep, Glob, Edit and Write existed.
  • Dry runs: eight local runs of the propose job's exact steps and flags over the 21 merges of 7 to 14 November, each 2 to 5 minutes.
    • Seven proposed the same patch, byte for byte, the last two of them with no cap on size: three dialog specs drop their stale copies of CdxDialogStub and import the shared one (+3/−23). With it, the 41 tests of those specs pass, and ESLint and vue-tsc -b are clean.
    • One run, made before the bar had its "Worth a review" condition, proposed a +2/−4 reuse of a test helper instead. All six later runs dropped that candidate.
  • AI review: a pr-review pass with code, security, design and text lenses found three bugs, all fixed. The pause matched anyone's simplify/ branch, an emptied new file was published as an empty file, and an abandoned edit could join the next proposal.
  • Built and reviewed on Opus 5.5 (max); the last changes to the bar and this description on Fable 5.2 (max).

AI-authored — Claude Code, Fable 5.2 (max), on a Professional Wiki account

Every Saturday at 02:00 UTC, Claude Code reads the past week's merges into master and proposes at most two
behaviour-preserving simplifications of the code they changed, each as a draft pull request. It targets what per-PR
review misses: duplication and leftovers that only show across PRs. While a proposal awaits review, the workflow
proposes nothing new. It files no issues and posts no comments. A manual run is a dry run: it only uploads the
patches.

The repository is public, so what the agent reads and writes is untrusted. It gets only file tools, confined to a
directory with the merges and the code, and nothing it writes runs in the workflow. A separate job holding the machine
account's PAT checks each patch and pushes it: only PHP, TypeScript, Vue and LESS files under `src/`,
`tests/phpunit/` and `resources/ext.neowiki/{src,tests}/`, no mode change, symlink or binary, fewer lines added than
removed, and at most 50 lines added and 300 changed. CI then runs the proposals, so the CI workflows now get a
read-only `GITHUB_TOKEN` instead of the repository default, which can write nearly everything. None of them needs
more, and none uses a secret.

The machine account, ProWikiExperts, needs write access to NeoWiki before the first scheduled run; it has read access.
The workflow uses the `CLAUDE_CODE_OAUTH_TOKEN` and `PR_TOKEN_PROWIKIEXPERTS` repository secrets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A proposal no longer has to stay under 50 added and 300 changed lines: it only has to remove more lines than it adds.
The caps were guesses that no proposal so far came near. A cap that is too tight fails unseen, as a warning on a run,
while an oversized pull request is seen and closed in seconds.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@alistair3149
alistair3149 merged commit 2b100bc into master Nov 19, 2026
23 checks passed
@alistair3149
alistair3149 deleted the ci/weekly-simplify branch November 19, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants