Skip to content

Add button-driven release panel and PyPI publish workflow - #14

Open
laxmareddyp wants to merge 1 commit into
keras-team:mainfrom
laxmareddyp:release-automation
Open

laxmareddyp wants to merge 1 commit into
keras-team:mainfrom
laxmareddyp:release-automation

Conversation

@laxmareddyp

Copy link
Copy Markdown
Contributor

Adds a centralized, button-driven release system for Keras repos. Only keras-hub is enabled; keras, keras-rs and kinetic have configs with enabled: false.

How it works

  • release.yml (Actions → release → Run workflow) is the panel. It works out the next step itself: cut the branch, cherry-pick, open a version-bump PR, then — after a human merges it and runs the panel again — create the tag and GitHub release in one POST /releases, and start publish.yml.
  • publish.yml validates the tag, builds, smoke-tests on TensorFlow, JAX and PyTorch, runs Gemini-generated tests, uploads to PyPI through trusted publishing, and checks that PyPI serves the same sha256 digests the build produced.
  • Dry run is on by default and reports branches, versions and candidate PRs.

Safety rules, each covered by a test

  • The bot never merges and only pushes release-tool/* branches. A static scan of the JS and of every workflow run:/script: body enforces this.
  • Anyone with write on the target repo may release; the App bot may act only for the person who started the panel.
  • A second release on the same branch is refused, not queued: active runs, open bump PRs and drafts on the line all count. Fork PRs are ignored.
  • Re-runs of the panel are refused; both workflows refuse to run from any ref but main.
  • Upload happens only if validate, build, smoke and the generated tests pass, or the generated tests failed and override_generated_tests was ticked.
  • The old per-repo publisher must be disabled before the panel publishes.

Verification (VM, Node 22.11 / Python 3.11)

  • node --test 'scripts/**/*.test.js': 225/225
  • pytest scripts/release/py: 49/49
  • zizmor 1.30.1 --offline on the three workflows: no findings
  • Mutation check: 25 guards deliberately broken one at a time; each turns the suite red.

Needs admin setup before first use (see RELEASE_PROCESS.md §8): the release GitHub App with secrets RELEASE_APP_CLIENT_ID / RELEASE_APP_PRIVATE_KEY in the release and release-read environments, repo variable RELEASE_APP_SLUG, GEMINI_API in the gemini environment, the PyPI trusted publisher, and the keras-hub rulesets.

Known follow-ups: check every package on PyPI (not only the first); show the full commit count in the "commits after merge" message; drop the hardcoded v tag prefix; dry-run wording when every pick is already on the branch. The install block in publish.yml is repeated on purpose — the smoke and generated-test jobs do not check out code.

@laxmareddyp

laxmareddyp commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor Author

This is a generated mockup, not a screenshot. The real form is GitHub's built-in workflow_dispatch form, drawn from the 11 inputs in release.yml; each field's grey helper text is that input's description.

image

@laxmareddyp

laxmareddyp commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Once automated releases are enabled across all Keras repos, I'll update the central release documentation and link it in the individual repos. As part of this cleanup, we’ll remove local publish-to-pypi.yml files, followed by Phase 2: centralizing all nightly workflows into this repo and removing them from all repos.

While the pipeline can run fully autonomously, we’re implementing two manual verification gates to prevent accidental PyPI publishes:

A bot opens a PR to create the release branch, builds the distribution artifacts, executes smoke tests, runs commit-based sanity checks, and drafts the release notes.

An explicit manual approval gate within the workflow authorizes the final publish to PyPI.

@laxmareddyp
laxmareddyp marked this pull request as draft September 28, 2026 18:44
@laxmareddyp
laxmareddyp removed the request for review from hertschuh September 28, 2026 18:45
@laxmareddyp
laxmareddyp marked this pull request as ready for review September 30, 2026 20:37
@jeffcarp

jeffcarp commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Thanks! This is quite a large PR. I'm not sure if we want to take on the burden of maintaining 7k+ LOC for release management. Can it be simplified? Here's a suggestion:

  1. Workflow 1 (prepare-release.yml): Takes repo, release_branch, version, optional commit_sha / cherry_picks, cuts the branch if needed, cherry-picks the commits,
  bumps version.py, and opens the PR(s).
  2. Workflow 2 (existing per-repo publish-to-pypi.yml, or a reusable workflow called on release: [published]): Runs pip_build.py, runs a quick multi-backend import
  smoke test on the built wheel, and uploads to PyPI via the repo's existing OIDC Trusted Publisher—no custom state machine, no run-title locking, no runtime Gemini
  test generation, and no moving PyPI trusted publishers out of the target repos.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants