Skip to content

feat(ledger): gate future Raft protocol capabilities - #2327

Open
gfyrag wants to merge 123 commits into
mainfrom
codex/en-3046-capabilities
Open

gfyrag wants to merge 123 commits into
mainfrom
codex/en-3046-capabilities

Conversation

@gfyrag

@gfyrag gfyrag commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • add backward-readable capability policy and replay-floor fields to the replicated cluster policy
  • declare stable required capability IDs on final Raft proposals and reject mismatched declarations before rawNode.Propose
  • add deterministic membership/policy evaluation for voters and learners, plus rollout and recovery documentation

Validation

  • bash scripts/agent-check
  • go test ./internal/infra/capabilities ./internal/infra/node ./internal/application/admission

Release sequencing

Relates to EN-3046. This PR is intentionally scoped to Ledger; the coordinated operator upgrade in EN-3047 is out of scope.

@NumaryBot NumaryBot added risk: high NumaryBot classified this pull request as high risk. bot-reviewed NumaryBot completed its review workflow for the current head. review-inconclusive NumaryBot could not prove that the review gate is satisfied. labels Oct 19, 2026
@gfyrag
gfyrag force-pushed the codex/en-3046-capabilities branch from d0f9127 to 5c93ce3 Compare October 19, 2026 15:35
@gfyrag

gfyrag commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

The Dirty check failed in run 39279211208 because just pre-commit rewrote six tracked files on the CI toolchain, then the final git diff check failed. The first mismatch was generated-source/toolchain drift (protoc v7.34.2 / protoc-gen-go-vtproto (devel) in CI), plus the nlreturn formatter changes in the capability files. I regenerated and formatted those exact outputs, verified the worktree is clean after just pre-commit up to the long-running lint stage, and pushed fix commit 9d595ca05 (chore(ledger): sync generated capability sources).

@gfyrag
gfyrag force-pushed the codex/en-3046-capabilities branch 4 times, most recently from e8fec16 to cf3473a Compare October 21, 2026 13:56

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

2 P2

The PR adds replicated capability-policy and proposal declaration fields plus pure validation helpers, then adds a pre-Raft declaration check. The admission barrier and replay floor are not fully enforced or preserved across normal policy updates and recovery.

Reviewed 34ab25b830cf3fb9c83ddc9662accd1b075b3248...cf3473a1ee663ce1ff6e2c5c914c4eb3902aaed0.

@gfyrag
gfyrag force-pushed the codex/en-3046-capabilities branch 2 times, most recently from 93d9f3d to 3282162 Compare October 21, 2026 15:02

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

New since cf3473a: 6 P1 · 1 P2

Resolved 2 earlier findings.

Reviewed the changes since cf3473a1ee663ce1ff6e2c5c914c4eb3902aaed0 in 9c57884775ae39fcff972e6af112fd1f197d61a6...32821622bc385bbb83539b7307a82bdc975bc84b.

Comment thread internal/bootstrap/module.go Resolved
Comment thread internal/infra/membership/membership.go Resolved
Comment thread internal/infra/node/node.go Resolved
Comment thread internal/infra/node/node.go Resolved
Comment thread internal/infra/membership/peer_store.go Resolved
Comment thread internal/infra/node/node.go Resolved
@gfyrag

gfyrag commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in commit 6d76261.

The failing Tests-E2E-Business, Tests-E2E-Cluster, Tests-Model, and Tests-Antithesis-Workload jobs all exposed the same regression: Admission.Barrier serialized a semantic proposal without declaring its required capabilities, so the final proposal validator rejected it. Barrier now stamps the semantic capability field before marshaling and copies the derived declaration to the Raft proposal. Added a regression test that validates the final wire declaration.

@gfyrag

gfyrag commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in commit e9899e8.

The Dirty job was caused by seven protobuf artifacts generated with a local protoc/vtproto toolchain different from CI. These artifacts were regenerated with nix develop .#ci and the diff is limited to the CI-compatible generated output.

@gfyrag

gfyrag commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in commit c3a8fd9.

The Dirty job also reported lint --fix drift in eight Go files. The CI lint formatting/import changes are now committed; just pre-commit completes cleanly with 0 issues.

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

New since 3282162: 1 P2

7 earlier findings are still open.

Reviewed the changes since 32821622bc385bbb83539b7307a82bdc975bc84b in 9c57884775ae39fcff972e6af112fd1f197d61a6...c3a8fd915a66caf89f1a4733c2f69fdb38b6af9a.

@gfyrag
gfyrag force-pushed the codex/en-3046-capabilities branch from c3a8fd9 to 635aee8 Compare October 22, 2026 09:14

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

No new issues in 1 file changed since c3a8fd9.

8 earlier findings are still open.

Reviewed the changes since c3a8fd915a66caf89f1a4733c2f69fdb38b6af9a in f42c74fe6c770d907fe10bf6de7ecdc5c6220a2a...1e1fe85246fd7390b2b65fa40ea673fd44840320.

@gfyrag
gfyrag force-pushed the codex/en-3046-capabilities branch 2 times, most recently from 1faef2b to c8ad6f7 Compare October 22, 2026 11:08

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

New since 1e1fe85: 1 P1 · 1 P2

Resolved 1 earlier finding. 7 earlier findings are still open.

Reviewed the changes since 1e1fe85246fd7390b2b65fa40ea673fd44840320 in 3d73377506b392655c48f898cf87c93fec86cceb...c8ad6f7899ad9106d93cbdd266abbdda0182067c.

Comment thread internal/bootstrap/module.go Resolved

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

New since c8ad6f7: 1 P1

Resolved 1 earlier finding. 8 earlier findings are still open.

Reviewed the changes since c8ad6f7899ad9106d93cbdd266abbdda0182067c in 3d73377506b392655c48f898cf87c93fec86cceb...149dfff81c7adc894e1c73e2df38cba2af835bf3.

Comment thread internal/bootstrap/module.go Resolved
@NumaryBot NumaryBot removed risk: high NumaryBot classified this pull request as high risk. bot-reviewed NumaryBot completed its review workflow for the current head. review-inconclusive NumaryBot could not prove that the review gate is satisfied. labels Oct 22, 2026
@gfyrag
gfyrag force-pushed the codex/en-3046-capabilities branch from a99d84c to c27bde1 Compare November 11, 2026 13:39

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

No new issues in 3 files changed since a99d84c.

Reviewed the changes since a99d84cc5b4e28b0e4fa200621633b6b130eb43e in 20c55fd8538630599b2dd03b3489454792263798...c27bde1e6a2a7c8a1b9a4e48a76d60ff0e30bcf2.

@gfyrag gfyrag added the perf-5m Request a 5-minute performance smoke benchmark label Nov 11, 2026

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

No new issues in 1 file changed since c27bde1.

Reviewed the changes since c27bde1e6a2a7c8a1b9a4e48a76d60ff0e30bcf2 in 20c55fd8538630599b2dd03b3489454792263798...cb1af52cfeafabc61ba909d5bd2bc118c3f014d5.

flemzord
flemzord previously approved these changes Nov 11, 2026

@flemzord flemzord left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DECISION: APPROVE
Head reviewed: cb1af52
Base: 20c55fd

Incremental re-review from 3a2d277, including the rebased full PR context. The range-diff preserves the previously reviewed implementation; the functional delta fixes startup and background RPC deadline handling and narrows activation detection to the dedicated policy payload emitted by admission. The generated protobuf follow-up changes only its compiler-version header.

FIXED: the remaining bounded restart-refresh finding. Both local attempt expiry and an early remote DeadlineExceeded remain transient while the lifecycle is live. Repeated remote deadlines now retry through success; lifecycle cancellation and permanent rejection remain terminal. The background retry and comment-contract findings are also fixed. The earlier predicted-index capacity finding remains fixed. Native activation extraction, cancellation-safe membership fencing, recovered-peer refresh, and unsupported-capability recovery checks remain intact.

Validation passed in pinned Nix: agent-check; capability contract; revision-40 wire compatibility; protobuf lint; complete race suites for bootstrap, admission, capabilities, node, infrastructure/application membership, plan, state, policy processing, and reconciler. Refresh/error-classification regressions passed 30 race-enabled repetitions. Focused race checks passed for gRPC, backup policy, protobuf JSON, and validation gate selection. The evaluator benchmark and full-PR whitespace check passed. Tracked files are unchanged.

Limits: agent-check-pr gate selection was inspected with the exact base; its broad/full, E2E, and normalization gates were not executed in this read-only review. Released-binary upgrades, an activation-specific incremental-restore delta, and complete admission latency were not exercised locally. No .codegraph directory is present; source and caller inspection used Git and rg. Immediately before publication, the PR was open, non-draft, conflict-free, and unchanged at the reviewed head. Repository Invariants passed; Default and PR Images are still running.

Blocking findings: 0
Non-blocking findings: 0
Residual risk: MEDIUM

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

New since cb1af52: 1 P2

Reviewed the changes since cb1af52cfeafabc61ba909d5bd2bc118c3f014d5 in 20c55fd8538630599b2dd03b3489454792263798...266722e8e42818bd25e44d31bb4b2b7c7c14f518.

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

No new issues in 11 files changed since 266722e.

Resolved 1 earlier finding.

Reviewed the changes since 266722e8e42818bd25e44d31bb4b2b7c7c14f518 in 20c55fd8538630599b2dd03b3489454792263798...190b172fc13ae92aa249820913f8af5cf7728736.

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

New since 190b172: 1 P2

Reviewed the changes since 190b172fc13ae92aa249820913f8af5cf7728736 in 20c55fd8538630599b2dd03b3489454792263798...72808378d58ade03ecbaf233e5b13b17f6c09564.

Comment thread internal/infra/membership/membership.go Resolved

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

No new issues in 1 file changed since 7280837.

Resolved 1 earlier finding.

Reviewed the changes since 72808378d58ade03ecbaf233e5b13b17f6c09564 in 20c55fd8538630599b2dd03b3489454792263798...0e1aa37e90a262e53ecfc2b75c67eab999168e5e.

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

No new issues in 2 files changed since 0e1aa37.

Reviewed the changes since 0e1aa37e90a262e53ecfc2b75c67eab999168e5e in 20c55fd8538630599b2dd03b3489454792263798...6f87a0c0fe80d0678b5a134da095985fc4f37846.

@shipfox-ai shipfox-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipfox review

No new issues in 1 file changed since 6f87a0c.

Reviewed the changes since 6f87a0c0fe80d0678b5a134da095985fc4f37846 in 20c55fd8538630599b2dd03b3489454792263798...55fd2b840b86562221b5a263c3a47bbc2471934b.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot-reviewed NumaryBot completed its review workflow for the current head. build-images perf-5m Request a 5-minute performance smoke benchmark review-inconclusive NumaryBot could not prove that the review gate is satisfied. risk: high NumaryBot classified this pull request as high risk.

Development

Successfully merging this pull request may close these issues.

5 participants