CI: publish the zenoh-flat-jni the snapshot was built against - #525
Conversation
|
Note from the review of eclipse-zenoh/zenoh-flat-jni#37: the qualifier and the commit stamp are produced by the
So merging this needs one extra step after #37 lands: cargo update -p zenoh-flat-jni --precise <#37 merge commit>Nothing else changes; the bot's lockfile sync moves it normally from then on. |
milyin
left a comment
milyin
left a comment
There was a problem hiding this comment.
Review of 4d718e7: I found two blockers and two additional issues.
-
P1 — Freeze the SDK source at the triggering commit.
.github/workflows/ci.yml:165passesbranch: ${{ github.ref_name }}, so the SDK publication resolvesmainonly after the JNI-copy workflow has spent roughly half an hour building. If commit B reachesmainwhile run A is active, concurrency queues B's run but does not stop the branch itself moving: run A read Cargo.lock and built the JNI pin from A, then its final publish job can check out and publish SDK source B against A's JNI copy. If B moved the pin, this recreates the JNI mismatch the PR is designed to prevent, and it can persist if B's queued run later fails. Pass the immutable event commit (github.sha, or an SDK SHA output captured beside the pin) to the reusable SDK publish workflow. -
P1 — The PR is not merge-ready until the upstream workflow lands and the pin moves. All four Actions attempts for this branch currently end in
startup_failurebecause zenoh-flat-jni #37's new reusable-workflow inputs are not on itsmainyet. After #37 merges, this PR still passes Cargo.lock'se75529c; that commit predates bothversionQualifierandzenoh.flatJniCommit, so #37's compatibility preflight intentionally rejects it. Merge #37 first, update Cargo.lock to a post-#37 commit, and rerun the complete CI workflow before merging this PR. Current failed run: https://github.com/eclipse-zenoh/zenoh-java/actions/runs/31519265662 -
P2 — Three matching POM stamps are not a completion marker.
ci/scripts/flat-jni-copy.bash:117-125setsrebuild=falseonce the three POM properties match, without checking the Gradle module metadata or native-bearing artifacts. The real upstream publication order demonstrates the failure window: the Android POM is uploaded before its.modulefile (and the root/JVM POMs likewise precede their module metadata). If the last publication fails after the Android POM but before its module metadata, the next run sees all three stamps, skips rebuilding forever, and the Linux-only consumer smoke test cannot detect the broken Android variant. Check the required.moduleplus JAR/AAR files, or publish/read a completion marker only after all three publications finish. -
P2 —
PUBLISHING.mdstill instructs operators to use the superseded snapshot path. The new section says this repository publishes its own1.9.0-java-SNAPSHOT, but lines 193 and 241-284 still say an empty input falls back to1.9.0and fails, and direct users to zenoh-flat-jni's rehearsal snapshot. Line 366 also lists “No consumer test” as a known gap even though this PR adds one. These are now contradictory release instructions and should be updated in the same change.
Validation performed: flat-jni-copy.bash --self-test passes; its live check correctly reports all three 1.9.0-java-SNAPSHOT stamps missing and rebuild=true for the current e75529c pin; git diff --check passes. No project CI jobs have run because the workflow currently fails during startup validation as noted above.
— Codex (GPT-5)
…whole Review of #525. - **The SDK publication was resolving `main`, not the run's commit.** Concurrency queues a newer run; it does not hold the branch still. So run A could read Cargo.lock at A, spend half an hour building the JNI copy from A's pin, and then publish SDK source B against it — recreating exactly the mismatch this job exists to prevent, and persisting if B's queued run later fails. It now passes `github.sha`. - **Three matching stamps were not proof of a finished publication.** The POM is uploaded before the Gradle module metadata and before the jar or aar, so a run that died in between would leave three readable stamps, no module metadata for a consumer to resolve a variant against, and a decision to skip rebuilding — permanently, since every later run reads the same three stamps. Nothing downstream would catch it: the consumer smoke test runs on Linux and the broken variant could be the Android one. The decision now also requires each coordinate's metadata to advertise `pom`, `module` and its binary (`jar`, or `aar` for Android), which is what a complete upload lists. Verified against the real 1.9.0-rc8-SNAPSHOT, and the two ways to be unfinished are in the self-test. - **PUBLISHING.md contradicted itself.** The rehearsal sections still said an empty `zenoh-flat-jni-version` falls back to an unreleased `1.9.0` and dies while compiling, and sent operators to zenoh-flat-jni's own rehearsal snapshot. The fallback is now our own `1.9.0-java-SNAPSHOT`, which resolves — what it cannot do is reach a live release, and it is `bump-and-tag.bash` that stops that. "No consumer test" was also listed as a known gap; the gap now is only that a *release* candidate is not consumable, being in staging.
|
1, 3 and 4 are fixed in fd17b7e; 2 is coordination and stands as you describe. 1 — the SDK source was not frozen. Right, and the distinction matters: concurrency queues the run, it does not hold the branch. 3 — three stamps are not a completion marker. Agreed, and the failure is worse than a one-off: every later run reads the same three stamps, so the skip is permanent, and the Linux smoke test cannot see a broken Android variant. The decision now also requires each coordinate's zenoh-flat-jni: complete
zenoh-flat-jni-jvm: complete
zenoh-flat-jni-android: completeand both ways of being unfinished are in I did not go for a completion marker: it needs an extra upload in zenoh-flat-jni's workflow, published after all three, and the metadata already carries the same information for the price of a request we make anyway. 4 — contradictory instructions. Fixed. Empty 2 — merge order. Confirmed, and the |
milyin
left a comment
milyin
left a comment
There was a problem hiding this comment.
Re-review of fd17b7e:
The immutable SDK checkout and the documentation corrections address findings 1 and 4 from the previous review. The new completeness check improves the first-publication case, but it still accepts the more important partial-overwrite case.
-
P1 — Compare snapshot generations, not only advertised extensions.
ci/scripts/flat-jni-copy.bash:76-91accepts metadata whenever it contains unclassifiedpom,module, andjar/aarentries. Those entries already exist after the first complete publication and remain present while a later snapshot is overwritten. Each<snapshotVersion>has its own<value>; during a failed overwrite, the POM entry can point to build N+1 while the module and binary entries still point to build N.advertisessees all three extensions,timestamped_namedownloads the N+1 POM, andpublished_commitreturns its new commit stamp. Every later run then skips rebuilding even though the coordinate is split across two JNI builds—the same permanent failure this check is meant to prevent. The real1.9.0-rc8-SNAPSHOTmetadata confirms that every extension carries an independent<value>. Require the unclassified POM, module, and primary binary values to equal the same${baseVersion}-${timestamp}-${buildNumber}(and preferably verify those files resolve), then add a self-test fixture with an N+1 POM plus N module/binary entries. The current tests cover missing entries only, so they cannot catch this replay case. -
P1 — The upstream/pin merge blocker is still outstanding. eclipse-zenoh/zenoh-flat-jni#37 remains open, and Cargo.lock still pins
e75529ce3758401ce213456e7b8e4e5667635cf8, whose Gradle build has neitherversionQualifiernorflatJniCommitsupport. Consequently both Actions attempts for this head still fail at workflow startup: https://github.com/eclipse-zenoh/zenoh-java/actions/runs/31528237094. After #37 merges, repin to a commit containing it and run the complete publication/consumer workflow before merging this PR.
Validation performed at fd17b7e: the updated flat-jni-copy.bash --self-test passes; the live check reports all three copies absent and rebuild=true; git diff --check passes. No project CI job has executed because reusable-workflow validation still fails before job creation.
— Codex (GPT-5)
Review of #525. The presence check missed the case that matters more than a first publication: an overwrite that failed part-way. Every <snapshotVersion> in a snapshot's maven-metadata.xml carries its own <value>, updated as that file lands. After the first complete publication all three extensions are listed and stay listed, so an overwrite that replaced the POM and then failed leaves the POM at build N+1 with the module metadata and the binary still at N. The old check saw three extensions, read the N+1 POM, found the pin in its stamp, and skipped rebuilding — permanently, and with a coordinate split across two JNI builds. So the check is now agreement rather than presence: the unclassified pom, module and binary entries must all name the same <timestamp>-<buildNumber>, and the POM is fetched by that name, so a metadata entry pointing at a file that never landed reads as no stamp and rebuilds. `timestamped_name` became `snapshot_value`, since the build identifier rather than one file name is what is being compared. The self-test gains the split case, which the previous fixtures could not express. Checked against the published 1.9.0-rc8-SNAPSHOT: all three coordinates whole, at 1.9.0-rc8-20260810.012355-1.
|
Fixed in baeb6ff. 1 — generations, not extensions. You are right, and the overwrite case is the one that matters: after the first complete publication all three extensions are listed and stay listed, so presence stops discriminating exactly when the coordinate starts being overwritten. A POM at N+1 with the module metadata and binary still at N passed the old check, and the stamp it then read was the new one — so the pin matched and the skip was permanent, over a coordinate split across two JNI builds. The check is now agreement. The unclassified all_at "$value" pom module "$3" <<<"$metadata" || return 0
The split fixture is in the self-test, which the previous ones could not express: # pom at N+1, module and jar still at N — every extension present
if all_at "$m" pom module jar <<<"…"; then
echo "all_at: accepted a coordinate split across two builds" >&2; exit 1
fiAnd against the real repository: zenoh-flat-jni: whole, at 1.9.0-rc8-20260810.012355-1
zenoh-flat-jni-jvm: whole, at 1.9.0-rc8-20260810.012355-1
zenoh-flat-jni-android: whole, at 1.9.0-rc8-20260810.012355-12 — merge blocker. Unchanged and acknowledged. eclipse-zenoh/zenoh-flat-jni#37 has no findings left as of its latest re-review, so the sequence is: merge #37 → |
milyin
left a comment
milyin
left a comment
There was a problem hiding this comment.
Re-review of baeb6ff:
The mixed-snapshot finding is resolved. all_at now requires the unclassified POM, Gradle module metadata, and primary JAR/AAR entries to carry the same snapshot <value> derived from the coordinate's current timestamp/build number. That rejects both an incomplete first publication and a partial overwrite that leaves build N+1's POM beside build N's module/binary. The new N/N+1 fixture exercises the previously missing failure mode, and the POM is fetched by the agreed value before its commit stamp is trusted. I found no new code-level findings in this update.
The PR is still not merge-ready for the previously noted external reason: eclipse-zenoh/zenoh-flat-jni#37 remains open, Cargo.lock still pins pre-#37 commit e75529ce3758401ce213456e7b8e4e5667635cf8, and both CI attempts for this head end in startup_failure: https://github.com/eclipse-zenoh/zenoh-java/actions/runs/31529401978. After #37 merges, repin zenoh-flat-jni to a commit containing its qualifier/stamp support and run the complete publication plus consumer workflow before merging.
Validation performed at baeb6ff: flat-jni-copy.bash --self-test passes; the live decision check safely reports all three copies absent and rebuild=true; git diff --check passes. No project CI job has executed yet because reusable-workflow validation fails before jobs are created.
— Codex (GPT-5)
Three things in the `on:` block, following the main zenoh repository as
zenoh-flat-jni#36 did.
`push` and `pull_request` were both on `["**"]`, so every push to a branch with
an open pull request ran the whole job set twice - two builds on two runners for
one commit, and two entries in the checks list to read. `push` now covers main
and `release/*`, minus the dry-run branches. A release is built from a branch the
shared create-release-branch action creates, and CI has never run on it -
`push: ["**"]` did cover that, but only as a side effect of covering everything.
`pull_request` stays on every branch, so a backport targeting a release branch
still gets CI, and nothing loses coverage: a branch under review is covered
there, and a branch that is not under review had no reader for its result.
The weekday nightly goes. The obvious argument for it is that it would catch
zenoh-flat-jni moving under us, and it does not: Cargo.lock pins that dependency
to a commit,
source = "git+https://github.com/eclipse-zenoh/zenoh-flat-jni.git?branch=main#<sha>"
and Cargo re-resolves a git dependency only on `cargo update` or a missing lock
entry, so a timed build rebuilds exactly what the last merge built. Upstream
drift arrives here as a lockfile-sync pull request, which runs CI like anything
else. What a nightly would still catch is an expired Central token or GPG key,
and only during a week with no merges at all - every merge already exercises
them. That is a thin canary against a daily publication of two coordinates, and
under #525 it also risks a half-hour zenoh-flat-jni rebuild for no new commit.
workflow_dispatch still runs the path on demand.
zenoh-flat-jni#36 dropped its own nightly for the same reason and recorded it in
PUBLISHING.md; its description said the other JVM repositories keep theirs, which
this makes stale - only zenoh-kotlin does now.
Concurrency is deliberately not touched here - #525 adds it, with
cancel-in-progress false, because the pair of publications that branch
introduces must not be cancelled between the two uploads.
…whole Review of #525. - **The SDK publication was resolving `main`, not the run's commit.** Concurrency queues a newer run; it does not hold the branch still. So run A could read Cargo.lock at A, spend half an hour building the JNI copy from A's pin, and then publish SDK source B against it — recreating exactly the mismatch this job exists to prevent, and persisting if B's queued run later fails. It now passes `github.sha`. - **Three matching stamps were not proof of a finished publication.** The POM is uploaded before the Gradle module metadata and before the jar or aar, so a run that died in between would leave three readable stamps, no module metadata for a consumer to resolve a variant against, and a decision to skip rebuilding — permanently, since every later run reads the same three stamps. Nothing downstream would catch it: the consumer smoke test runs on Linux and the broken variant could be the Android one. The decision now also requires each coordinate's metadata to advertise `pom`, `module` and its binary (`jar`, or `aar` for Android), which is what a complete upload lists. Verified against the real 1.9.0-rc8-SNAPSHOT, and the two ways to be unfinished are in the self-test. - **PUBLISHING.md contradicted itself.** The rehearsal sections still said an empty `zenoh-flat-jni-version` falls back to an unreleased `1.9.0` and dies while compiling, and sent operators to zenoh-flat-jni's own rehearsal snapshot. The fallback is now our own `1.9.0-java-SNAPSHOT`, which resolves — what it cannot do is reach a live release, and it is `bump-and-tag.bash` that stops that. "No consumer test" was also listed as a known gap; the gap now is only that a *release* candidate is not consumable, being in staging.
Review of #525. The presence check missed the case that matters more than a first publication: an overwrite that failed part-way. Every <snapshotVersion> in a snapshot's maven-metadata.xml carries its own <value>, updated as that file lands. After the first complete publication all three extensions are listed and stay listed, so an overwrite that replaced the POM and then failed leaves the POM at build N+1 with the module metadata and the binary still at N. The old check saw three extensions, read the N+1 POM, found the pin in its stamp, and skipped rebuilding — permanently, and with a coordinate split across two JNI builds. So the check is now agreement rather than presence: the unclassified pom, module and binary entries must all name the same <timestamp>-<buildNumber>, and the POM is fetched by that name, so a metadata entry pointing at a file that never landed reads as no stamp and rebuilds. `timestamped_name` became `snapshot_value`, since the build identifier rather than one file name is what is being compared. The self-test gains the split case, which the previous fixtures could not express. Checked against the published 1.9.0-rc8-SNAPSHOT: all three coordinates whole, at 1.9.0-rc8-20260810.012355-1.
aaa4bae to
c1b4588
Compare
) * CI: pin every third-party action to a commit zenoh-flat-jni#36 applied this rule there, from @diogomatsubara's review, and this repository is the other half of the same release: a tag is mutable, so a moved tag runs code nobody reviewed - on jobs that hold the organization GPG key and the Central token. Every third-party `uses:` across the four workflows now names a commit, with the version in a trailing comment. Actions under eclipse-zenoh/ are ours and stay on a branch, deliberately. The pins are the same commits zenoh-flat-jni landed, so the two repositories run the same action code. Two of them are also bumps, and each says why in place: - peaceiris/actions-gh-pages v3 -> v4.1.0. v3 runs on node16, a runtime the Actions runner has retired. Its inputs are unchanged. - gradle/actions/setup-gradle v4 -> v5, not v6: v6 moved caching into a proprietary component under Gradle's own terms of use, which is not ours to accept for an Eclipse project. markdownlint-cli2-action is pinned where it stands, at v18. It is on node20, which is not retired, and a newer major changes markdownlint's rules - that is an upgrade to make on its own, with whatever README edits it asks for, not inside a pinning change. checkout, setup-java and upload-artifact land on current majors as a consequence of pinning to those commits. Every input these workflows pass still exists in the pinned versions, and checkout still defaults persist-credentials to true, which the release branch push in ci/scripts/bump-and-tag.bash depends on. * CI: fix the triggers - no double runs, release branches, no nightly Three things in the `on:` block, following the main zenoh repository as zenoh-flat-jni#36 did. `push` and `pull_request` were both on `["**"]`, so every push to a branch with an open pull request ran the whole job set twice - two builds on two runners for one commit, and two entries in the checks list to read. `push` now covers main and `release/*`, minus the dry-run branches. A release is built from a branch the shared create-release-branch action creates, and CI has never run on it - `push: ["**"]` did cover that, but only as a side effect of covering everything. `pull_request` stays on every branch, so a backport targeting a release branch still gets CI, and nothing loses coverage: a branch under review is covered there, and a branch that is not under review had no reader for its result. The weekday nightly goes. The obvious argument for it is that it would catch zenoh-flat-jni moving under us, and it does not: Cargo.lock pins that dependency to a commit, source = "git+https://github.com/eclipse-zenoh/zenoh-flat-jni.git?branch=main#<sha>" and Cargo re-resolves a git dependency only on `cargo update` or a missing lock entry, so a timed build rebuilds exactly what the last merge built. Upstream drift arrives here as a lockfile-sync pull request, which runs CI like anything else. What a nightly would still catch is an expired Central token or GPG key, and only during a week with no merges at all - every merge already exercises them. That is a thin canary against a daily publication of two coordinates, and under #525 it also risks a half-hour zenoh-flat-jni rebuild for no new commit. workflow_dispatch still runs the path on demand. zenoh-flat-jni#36 dropped its own nightly for the same reason and recorded it in PUBLISHING.md; its description said the other JVM repositories keep theirs, which this makes stale - only zenoh-kotlin does now. Concurrency is deliberately not touched here - #525 adds it, with cancel-in-progress false, because the pair of publications that branch introduces must not be cancelled between the two uploads. * CI: dispatch-only releases, no nightly dry run either release.yml ran on `0 0 * * 1-5` as well as on dispatch. A scheduled run passes no inputs, so it passes no `zenoh-flat-jni-version`, so it fell back to `zenohFlatJniVersion` in gradle.properties - a version not on Maven Central - and died while compiling. Every weeknight, for as long as that has been true. PUBLISHING.md documented the failure rather than the schedule being wrong. Nothing is lost by removing it. A dry-run release rehearses the release path, and rehearsing is a deliberate act with a version in the box, exactly like the release it rehearses. Actions -> Release -> Run workflow does both. Same reasoning as the CI nightly in the previous commit, one workflow over.
The nightly snapshot publication has never got past compiling: it resolves `zenoh-flat-jni:1.9.0`, and nothing of zenoh-flat-jni has ever been published under any version. Naming its own `1.9.0-SNAPSHOT` instead would fix the symptom and break two things — this repository's CI would then wait on that repository's CI, and the coordinate always holds the tip of *its* main while we compile against the commit `Cargo.lock` pins. JNI being a binary contract, that mismatch surfaces as `UnsatisfiedLinkError` at runtime, not as a build failure. So the publication publishes what it depends on. On `main` — every merge plus the weekday nightly, as before — it builds zenoh-flat-jni from the pinned commit and uploads it as `1.9.0-java-SNAPSHOT`, then builds and uploads the SDK against that. Self-sufficient, because it uses that repository's *source at a commit we choose*, never an artifact its CI produced; coherent, because the dependency our POM names is what we compiled against. - `flat_jni_pin` reads the pin from `Cargo.lock` and the commit stamp from the three published coordinates. Rebuilding means ten cross-compiled targets and half an hour, and the pin moves about once a day, so it happens only when the published copy is not already that commit. Anything missing or unreadable reads as "not ours" and rebuilds. - `publish_flat_jni_copy` calls zenoh-flat-jni's own publication workflow rather than duplicating its build matrix. `uses:` cannot hold an expression, so the workflow file comes from its `main` and the pin goes in `branch:` — the coupling is to a file in that repository, never to a run of its CI. - `-java` keeps our copy from overwriting the one zenoh-flat-jni publishes or zenoh-kotlin's; the three can legitimately pin different commits at once. The name is fixed, so it is overwritten rather than accumulated. - `--refresh-dependencies` on both SDK invocations: Gradle caches changing modules for 24 hours and setup-gradle restores that cache, so without it the SDK could compile against yesterday's copy while publishing a POM naming the coordinate that now holds today's. - `ci/consumer-smoke-test` resolves the published snapshot from a clean build with no connection to this one and takes a key expression through JNI. That is the only check that the POM, the transitive dependency and the native library work for someone who is not us. - `bump-and-tag.bash` now checks the value `gradle.properties` ends up with: main inherits a snapshot, so *omitting* `zenoh-flat-jni-version` is how a release would reach one, which the input-only check did not cover. - CI runs on `main` are serialized rather than cancelled. Two uploads cannot be made atomic, and cancelling mid-publication is what splits them; PUBLISHING.md says so rather than claiming coherence by construction. The test job is untouched: it still builds zenoh-flat-jni from source through the composite build, which keeps that path from rotting. Verified end to end locally through Maven Local: the qualified copy publishes with its commit stamp, the SDK compiles and publishes against it, and an outside consumer resolves both and prints `zenoh-java smoke test OK`. Needs eclipse-zenoh/zenoh-flat-jni's `source-repository`/`version-qualifier` inputs on `main` first.
Its coordinate comes from its own version.txt, ours from gradle.properties. A pin that moves past a version bump there would publish 1.10.0-java-SNAPSHOT while this SDK still resolves 1.9.0-java-SNAPSHOT — and only fail afterwards. flat-jni-copy.bash now emits the base version and ci.yml passes it as expected-base-version, so eclipse-zenoh/zenoh-flat-jni#37's preflight catches the mismatch before anything is built.
…whole Review of #525. - **The SDK publication was resolving `main`, not the run's commit.** Concurrency queues a newer run; it does not hold the branch still. So run A could read Cargo.lock at A, spend half an hour building the JNI copy from A's pin, and then publish SDK source B against it — recreating exactly the mismatch this job exists to prevent, and persisting if B's queued run later fails. It now passes `github.sha`. - **Three matching stamps were not proof of a finished publication.** The POM is uploaded before the Gradle module metadata and before the jar or aar, so a run that died in between would leave three readable stamps, no module metadata for a consumer to resolve a variant against, and a decision to skip rebuilding — permanently, since every later run reads the same three stamps. Nothing downstream would catch it: the consumer smoke test runs on Linux and the broken variant could be the Android one. The decision now also requires each coordinate's metadata to advertise `pom`, `module` and its binary (`jar`, or `aar` for Android), which is what a complete upload lists. Verified against the real 1.9.0-rc8-SNAPSHOT, and the two ways to be unfinished are in the self-test. - **PUBLISHING.md contradicted itself.** The rehearsal sections still said an empty `zenoh-flat-jni-version` falls back to an unreleased `1.9.0` and dies while compiling, and sent operators to zenoh-flat-jni's own rehearsal snapshot. The fallback is now our own `1.9.0-java-SNAPSHOT`, which resolves — what it cannot do is reach a live release, and it is `bump-and-tag.bash` that stops that. "No consumer test" was also listed as a known gap; the gap now is only that a *release* candidate is not consumable, being in staging.
Review of #525. The presence check missed the case that matters more than a first publication: an overwrite that failed part-way. Every <snapshotVersion> in a snapshot's maven-metadata.xml carries its own <value>, updated as that file lands. After the first complete publication all three extensions are listed and stay listed, so an overwrite that replaced the POM and then failed leaves the POM at build N+1 with the module metadata and the binary still at N. The old check saw three extensions, read the N+1 POM, found the pin in its stamp, and skipped rebuilding — permanently, and with a coordinate split across two JNI builds. So the check is now agreement rather than presence: the unclassified pom, module and binary entries must all name the same <timestamp>-<buildNumber>, and the POM is fetched by that name, so a metadata entry pointing at a file that never landed reads as no stamp and rebuilds. `timestamped_name` became `snapshot_value`, since the build identifier rather than one file name is what is being compared. The self-test gains the split case, which the previous fixtures could not express. Checked against the published 1.9.0-rc8-SNAPSHOT: all three coordinates whole, at 1.9.0-rc8-20260810.012355-1.
#526 removes both weekday schedules - CI's 06:00 snapshot run and release.yml's 00:00 dry run - so three sentences added here describe a trigger that will not exist: - the ci.yml comment above the publishing jobs, "every merge there plus the weekday nightly above"; - "every merge to `main` and the weekday nightly upload a mutable pre-release build"; - the rehearsal section, which offered the nightly release run as an example of leaving `zenoh-flat-jni-version` empty. The last one was the weakest of the three anyway: that run failed while compiling every night, because an empty input fell back to a version not on Maven Central. What this branch changes about it is the part worth saying - the fallback is now our own published copy, so an empty field is a sound default rather than a guaranteed failure. The crons themselves are left to #526; touching them here would only conflict.
#526 pins every third-party action in these workflows; the four steps added here - the checkout in flat_jni_pin, and the checkout, setup-java and setup-gradle in consumer_test - were written against the unpinned form and would land on main as the only tags left. Same commits, same trailing comments.
eclipse-zenoh/zenoh-flat-jni#37 is merged, so `publish.yml@main` now accepts `source-repository`, `version-qualifier` and `expected-base-version`, and every POM it publishes carries `zenoh.flatJniCommit`. The pin here was e75529c, which predates all of that: the preflight added by that PR rejects it by design, and the workflow this branch adds could not start at all while the inputs it names were not on main. Only the zenoh-flat-jni line moves. `cargo update -p zenoh-flat-jni --precise` also re-resolved the zenoh git dependencies to the tip of their branch, which this branch has no business carrying: zenoh-flat-jni@6b5c04c was tested against 773126fd, and the lockfile-sync bot is what moves that rev in step with Zenoh's own. They are pinned back, leaving a one-line diff.
e30c72c to
b7a2984
Compare
* CI: pin third-party actions, fix the triggers, drop both nightlies The same three changes eclipse-zenoh/zenoh-java#526 made, for the same reasons. - Every third-party action pinned to a commit, version in a trailing comment. A tag is mutable, and these workflows hold the signing key and the Central token. eclipse-zenoh/* actions are ours and stay on a branch. Pinning also picked up stale majors: setup-java v4 -> v5.7.0, setup-gradle v4 -> v5.0.2, upload-artifact v4 -> v7.0.1, actions-gh-pages v3 -> v4.1.0. setup-gradle stops at v5: v6 moved caching into a proprietary component under Gradle's own terms of use, which is not ours to accept for an Eclipse project. - Triggers follow the main zenoh repository. `push` on main and release branches - a release is built from a branch create-release-branch makes, and CI never ran on it - and `pull_request` on every branch, so a backport against a release branch gets CI. Not `push` on every branch as well: with pull_request on, that ran the whole matrix twice per branch. - Both nightlies go. ci.yml's would have rebuilt exactly what the last merge built, because Cargo.lock pins zenoh-flat-jni to a commit and Cargo re-resolves a git dependency only on `cargo update` - upstream drift arrives as a lockfile-sync pull request instead. release.yml's was worse than useless: a scheduled run passes no inputs, so every night it resolved the unreleased fallback in gradle.properties and died compiling. Releases and rehearsals are both deliberate acts, and both are `Run workflow`. * CI: publish the zenoh-flat-jni the snapshot was built against Closes the zenoh-kotlin half of eclipse-zenoh/zenoh-java#524, the same way eclipse-zenoh/zenoh-java#525 closes the other. gradle.properties named `zenoh-flat-jni:1.9.0`, a version that exists in no form, so the snapshot publication died in compileKotlinJvm every time it ran. Naming zenoh-flat-jni's own `1.9.0-SNAPSHOT` instead would fix the symptom and break two things: this repository's CI would wait on that repository's CI, and that coordinate always holds the tip of *its* main while this SDK compiles against the commit Cargo.lock pins. JNI being a binary contract, the mismatch surfaces as UnsatisfiedLinkError at runtime rather than as a build failure. The rule instead: **the publication publishes what it depends on.** On main, on every merge, build zenoh-flat-jni from the pinned commit, upload it as 1.9.0-kotlin-SNAPSHOT, then build and upload the SDK against that. - `flat_jni_pin` reads the pin from Cargo.lock and the commit stamp from all three published coordinates. Rebuilding is ten cross-compiled targets and about half an hour, and the pin moves roughly once a day, so it happens only when the published copy is not already that commit. Anything missing or unreadable reads as "not ours" and rebuilds - the safe direction. - `publish_flat_jni_copy` calls zenoh-flat-jni's own publication workflow rather than duplicating its six-target/four-ABI matrix, which is how the two would drift. - `-kotlin` keeps our copy from overwriting the one zenoh-flat-jni publishes or zenoh-java's; the three can legitimately pin different commits at once. The name is fixed, so it is overwritten rather than accumulated. - `--refresh-dependencies` on both SDK invocations. Gradle caches changing modules for 24 hours and setup-gradle restores that cache, so without it a run could upload copy B, compile against cached copy A, and publish a POM naming the coordinate that now resolves to B. - `ci/consumer-smoke-test` - a separate Gradle build with no path, project or composite connection to this one. It resolves the published snapshot from the snapshot repository and takes a key expression through JNI. Kotlin rather than Java because KeyExpr.tryFrom returns a Result, which an inline value class makes awkward to call from Java. - `concurrency` with cancel-in-progress false. The two uploads are not atomic, and cancelling a run mid-publication is exactly what leaves them naming different commits. - The release guard in bump-and-tag.bash now checks the value gradle.properties ends up with rather than the workflow input, because main inherits a snapshot and omitting the input is how a release would reach one. The pin moves to 6b5c04c, eclipse-zenoh/zenoh-flat-jni#37's merge commit: the earlier e75529c predates the version-qualifier inputs and the commit stamp this depends on, and that repository's preflight rejects it by design. Only that line of Cargo.lock moves - `cargo update -p` also re-resolved the zenoh git dependencies to their branch tip, which is the lockfile-sync bot's job, so they are pinned back.
1 participant
Closes #524.
The failure
The nightly snapshot publication has never got past compiling —
run 31464535324:
gradle.propertiesnames a version that exists in no form; nothing ofzenoh-flat-jnihas ever been published.Naming its own
1.9.0-SNAPSHOTinstead (#523) fixes the symptom and breaks twothings. This repository's CI would wait on that repository's CI. And that
coordinate always holds the tip of its
main, while this SDK compiles againstthe commit
Cargo.lockpins — JNI being a binary contract, the mismatchsurfaces as
UnsatisfiedLinkErrorat runtime rather than as a build failure.The rule
The publication publishes what it depends on. On
main, on every merge, itbuilds
zenoh-flat-jnifrom the pinned commit, uploads it as1.9.0-java-SNAPSHOT, then builds and uploads the SDK against that.choose, never an artifact its CI produced. If zenoh-flat-jni's CI were
switched off entirely, this still works.
main, and the weekday nightlyzenoh-flat-jnicopy; then publish the SDK snapshot; then consume it as an outsiderzenoh-flat-jnifrom Maven CentralWhat changes
flat_jni_pinreads the pin fromCargo.lockand the commit stamp fromall three published coordinates. Rebuilding is ten cross-compiled targets and
about half an hour, and the pin moves roughly once a day, so it happens only
when the published copy is not already that commit. Anything missing or
unreadable reads as "not ours" and rebuilds — the safe direction.
publish_flat_jni_copycalls zenoh-flat-jni's own publication workflowrather than duplicating its six-target/four-ABI matrix, which is how the two
would drift.
-javakeeps our copy from overwriting the one zenoh-flat-jni publishesor zenoh-kotlin's; the three can legitimately pin different commits at once.
The name is fixed, so it is overwritten rather than accumulated.
--refresh-dependencieson both SDK invocations. Gradle caches changingmodules for 24 hours and
setup-gradlerestores that cache, so without it arun could upload copy B, compile against cached copy A, and publish a POM
naming the coordinate that now resolves to B.
ci/consumer-smoke-test— a separate Gradle build with no path, project orcomposite connection to this one. It resolves the published snapshot from the
snapshot repository and takes a key expression through JNI. It is the only
check that the POM, the transitive
zenoh-flat-jniand the native librarywork for someone who is not us.
bump-and-tag.bashnow checks the valuegradle.propertiesends up with,not the workflow input:
maininherits a snapshot, so omittingzenoh-flat-jni-versionis exactly how a release would reach one.main. Two uploads cannot be made atomic, andcancelling a run mid-publication is what leaves them naming different commits,
so
cancel-in-progress: falserather than thetruethe issue proposed.The test job is untouched — still
jvmTest -PuseLocalJni=truethrough thecomposite build, which keeps the local development path from rotting.
Three deviations from the plan in #524
uses: …@main, not a literal SHA.uses:cannot hold an expression, sothe pin cannot go there; and a SHA that does not exist yet cannot be reviewed.
The workflow file comes from that repository's
main, reviewed like anydependency, and the pin rides in
branch:. The coupling is to a file, neverto a run of its CI, which is what requirement 2 asks.
both uploads last; the copy is uploaded at the end of the called workflow and
the SDK afterwards. Closing that needs zenoh-flat-jni's workflow split into
stage and upload phases. Not done — PUBLISHING.md states best-effort coherence
instead of claiming coherence by construction.
Also corrected in the docs: fixed snapshot names are constant, but Central
still stores timestamped builds behind them and cleans them after 90 days.
Verification
End to end through Maven Local, which is the whole chain minus the upload:
The decision script's parsers have a self-test, and the read was checked against
the one snapshot that really exists —
1.9.0-rc8-SNAPSHOT, published before thestamp, so it correctly reports
stamp is noneand would rebuild:Resolution wiring, before and after publication exists:
Bootstrapping
1.9.0-java-SNAPSHOTdoes not exist until the first run onmainpublishes it,and that run publishes it before it needs it — so the first merge bootstraps
itself. Until then a contributor's plain
./gradlew buildcannot resolve it;-PuseLocalJni=trueand-PlocalJniDir=<path>are unaffected, and README saysso.
Merge order
After eclipse-zenoh/zenoh-flat-jni#37, which adds the
source-repositoryandversion-qualifierinputs this calls.Supersedes the
zenohFlatJniVersionhalf of #523; its release guard is kepthere, strengthened to check the effective value.
zenoh-kotlin gets the same change once this is green.
Stacked on #526
Base is
ci/pin-actions, notmain, so the diff above is this branch's ownchanges only; GitHub retargets it to
mainwhen #526 merges. Merge order:both weekday schedules (CI's 06:00 snapshot run, and
release.yml's 00:00dry-run release, which failed every night on the very error this PR closes).
flat_jni_pincheckout, andconsumer_test's checkout / setup-java / setup-gradle) are pinned like therest, so
maingets nothing unpinned;nightly release run as an example of leaving
zenoh-flat-jni-versionemptynow says what this branch actually changes about it: the fallback is our own
published copy, so an empty field is a sound default rather than the
guaranteed compile failure it used to be.
The
concurrencyblock stays as written here —cancel-in-progress: false,because the pair of uploads must not be cancelled between them. #526 deliberately
left it to this branch.