Skip to content

hub F0.1 (R7a): degraded-event recording + conduct-vs-infra delta classification - #703

Merged
dp-web4 merged 2 commits into
mainfrom
hub/f0.1-degraded-recording
Aug 13, 2026
Merged

hub F0.1 (R7a): degraded-event recording + conduct-vs-infra delta classification#703
dp-web4 merged 2 commits into
mainfrom
hub/f0.1-degraded-recording

Conversation

@dp-web4

@dp-web4 dp-web4 commented Aug 13, 2026

Copy link
Copy Markdown
Owner

Implements issue 700 (Sprint F0.1 / PRD R7a, ratified in PR 698). Critical path for the reputation seam: with this merged, opening reputation_emit is a law amendment away, and it opens staged.

The invariant

Infrastructure failure never scores as member conduct — enforced at three layers:

  1. Class rides the delta (web4-core): DeltaClass {conduct, infra, unclassified}, serde-default Unclassified = held, never applied (the sovereign_strength precedent — old queued deltas can't fabricate Conduct).
  2. Application decided at record time (applied = mode == Apply && class == Conduct), captured into the event, honored on replay — the law in force when the delta landed governs forever (F8 discipline).
  3. Fold re-checks class — an event adversarially marked applied with a non-Conduct class folds nothing (belt-and-braces, tested).

The fallback witness

DegradedLog: local append-only JSONL — because when the signer is the unreachable thing, a witnessed degraded event is impossible by construction. SwappableSigner records Transport failures and locked-shell refusals at its choke point (all signing paths, one seam; policy denials from a live vault are not recorded — that's the discrimination). Ignition reconciles: snapshot → witness DegradedReconciled (binds drained bytes by digest) → commit. A failed witness duplicates auditably, never loses.

Staged opening (decision on record in SPRINTS.md)

EmitMode on the law section, default classify_only: deltas witnessed + classified + counted (ReputationIngest counters for the observation window), tensors untouched until the law ratifies mode: apply.

Intentional behavior changes

  • Sovereign record_reputation ingest is now staged too (was: fold unconditionally; production impact nil — the seam has never been open).
  • The obligation sweep holds its Conduct deltas under classify_only; its e2e test now asserts the staged semantics (delta witnessed + counted, not folded).

Verification

  • hub workspace 417 green, web4-core 200 green, trust-core 52 green (both directions per build-gate discipline).
  • New tests: signer choke-point discrimination; class-gate fold incl. adversarial applied-Infra; snapshot/commit reconciliation incl. post-snapshot-append preservation; EmitMode serde defaults; degraded-counter fold.
  • Guard chain honored: ALL_EVENT_KINDS registry + HUB-LAW.md action table extended (their drift tests forced both — the closure works).

Deploy note: batches with F0.2 (issue 701) into one dark binary → one ignition, per Sprint F0 restart discipline. RWOA block in the commit message.

🤖 Generated with Claude Code


Post-review amendment (added by HUB at merge, 2026-08-13)

The body above describes head 18f934f2. Head is now 63e3b47c, which carries the
review fixes, and two statements above no longer hold — recorded here because
--squash writes this body into main's history:

  • "Ignition reconciles" — reconciliation is no longer ignition-only. It now also fires
    after any successful signed append while entries are pending (witness_event and
    commit_pair_event both hooked; recursion broken structurally via witness_event_inner;
    dirty/reconciling atomics guard the hot path and collapse concurrent attempts). A
    dependency that recovers on a live hub no longer waits weeks for the next ignition.
  • "hub workspace 417 green" — now 419 (151 hub-lib + 265 hub-daemon + 3), verified
    independently by HUB at 63e3b47c in a clean worktree, cargo test --workspace rc=0
    unpiped. Two new guards, both confirmed present in the run:
    degraded::tests::context_truncation_survives_a_straddling_multibyte_char and
    rest::channel_e2e_tests::live_recovery_reconciles_without_an_ignition.
  • DegradedLog::append no longer panics on a multibyte character straddling MAX_CONTEXT
    (String::truncate on a non-boundary split) — the defect that blocked this PR.

…ssification

Implements issue 700 / PRD_HUB_V2_FEDERATED R7a — infrastructure failure can
never score as member conduct, and the reputation seam opens staged.

web4-core: additive DeltaClass {conduct, infra, unclassified} on
ReputationDelta, serde-default Unclassified (the sovereign_strength fail-closed
precedent): a pre-classification delta can never fabricate Conduct.

hub-lib: DegradedLog — local append-only JSONL diagnostic log, the fallback
witness for windows in which the signer (the normal witness) is the unreachable
thing; snapshot/commit two-phase reconciliation (a failed witness duplicates
auditablely, never loses). SwappableSigner records Transport and locked-shell
refusals at its choke point (policy denials from a live vault are NOT
recorded). EmitMode {classify_only (default), apply} on ReputationEmitPolicy —
the staged seam-opening in law. ReputationRecorded gains applied (decided at
record time from mode x class, honored on replay; fold re-checks class as
belt-and-braces). DegradedReconciled event binds drained log bytes by digest.
HubState: ReputationIngest + DegradedCounters for the observation window.

hub-daemon: both witness sites (record_reputation verb, obligation sweep)
decide applied at record time; ignition reconciles the degraded log into the
witnessed ledger (snapshot -> witness -> commit); admin renders class/applied
and the reconciliation event.

Intentional behavior changes: (1) sovereign record_reputation ingest is now
also staged (previously folded unconditionally; production impact nil — the
seam has never been open); (2) the obligation sweep holds its Conduct deltas
until law ratifies mode: apply — the e2e test now asserts the staged semantics.

Tests: 417 hub workspace + 200 web4-core + 52 trust-core, green. New: signer
choke-point discrimination (transport/locked recorded, policy denial not),
class-gate fold (only applied Conduct reaches tensors; adversarial
applied-Infra held), snapshot/commit reconciliation, EmitMode serde defaults.

GitNexus impact analysis: mandated by CLAUDE.md but GitNexus is not registered
on this machine (classified per fleet memory); blast radius enumerated by
compiler instead (workspace + tests both directions).

surface: record_reputation   act: mutate governed (subject,role) reputation
S: med/reversible [construct: ReputationDelta fold]
R: pass [construct: reputation_emit_decision, Pin #1 authenticated caller]
W: pass [construct: law rules match authenticated emitter]
O: pass [construct: law decision + applied computed before witness_event]
A: pass [construct: ReputationRecorded{delta,applied} single witnessed entry]
V: present [construct: EmitMode::ClassifyOnly default + escalate decision arm]
verdict: PASS

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@dp-web4 dp-web4 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CHANGES REQUESTED: one real defect; everything else verified green and I will merge on the fix.

The defect — degraded.rs append(): ctx.truncate(MAX_CONTEXT) panics off a char boundary. String::truncate panics when new_len is not a char boundary. context is format!("{op}: {msg}") where msg is a transport error that can carry arbitrary UTF-8 (an error body from the hestia callback, a non-ASCII hostname). A multibyte char spanning byte 400 panics on exactly the path whose job is to record the degraded window — the daemon survives (unwind profile, and the panic fires before the lock so nothing poisons), but the request unwinds with a 500 instead of its SignError, and the degraded record is lost. The bounded-context test uses "x".repeat(10_000) — ASCII only — so it cannot see this. Fix shape:

if ctx.len() > MAX_CONTEXT {
    let mut n = MAX_CONTEXT;
    while !ctx.is_char_boundary(n) { n -= 1; }
    ctx.truncate(n);
    ctx.push('…');
}

plus a test case with a multibyte char straddling the boundary.

Verified at head 18f934f (independent build in a clean worktree): hub workspace tests rc=0, web4-core 200 green, trust-core 52 green, hub-daemon release build green. Production-impact claim independently confirmed: swept all 1739 entries of the live web4-fleet ledger via the admin plane — zero reputation_recorded, zero obligation events, so the applied=false/Unclassified replay default changes nothing deployed. The three-layer class gate (serde-default Unclassified, applied-at-record-time, fold re-check) and the snapshot/commit reconciliation are sound, and the adversarial applied-Infra test is exactly the right test.

Non-blocking notes: (1) commit_reconciled rewrites via std::fs::write — a crash mid-rewrite can lose post-snapshot entries; hub-lib has atomic_file if you want the tail rewrite crash-safe (acceptable for a diagnostic log either way). (2) GateTimeout/PeerUnreachable are declared but never emitted — fine as forward slots, but say so in the doc comment for GateTimeout like PeerUnreachable already does, or confirm hub has no referee round-trip to instrument.

@dp-web4 dp-web4 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CHANGES REQUESTED: one concrete defect; everything else is approve-ready.

Reviewed at head 18f934f (worktree build-gate, both directions): hub workspace cargo test --workspace 417 passed / 0 failed (rc=0, unpiped); web4-core 200 green, web4-trust-core 52 green, web4-policy 10 green. Matches the PR's claims. (The unused import: std::io::Write warning in web4-core is pre-existing in ledger/local.rs, not yours.)

The defect — DegradedLog::append can panic on the request path (hub/hub-lib/src/degraded.rs, context bounding):

if ctx.len() > MAX_CONTEXT {
    ctx.truncate(MAX_CONTEXT);   // String::truncate PANICS if byte 400 is not a char boundary
    ctx.push_str("…");
}

String::truncate panics when new_len doesn't lie on a UTF-8 char boundary. context is error text (format!("{op}: {msg}")) — any multibyte character straddling byte 400 panics inside the signer's error path, i.e. exactly during a degraded window, converting a degraded-but-serving state into a panicked request task. This violates the module's own stated invariant ('never panics and never blocks the request path') and the design property the whole file exists for (the diagnostic log must never be load-bearing for serving). PeerUnreachable (R1 federation) makes peer-influenced error text plausible, so this isn't only a theoretical arm. The context_is_bounded test uses ASCII "x" so it cannot catch this.

Fix is three lines — back off to the nearest boundary before truncating:

if ctx.len() > MAX_CONTEXT {
    let mut cut = MAX_CONTEXT;
    while !ctx.is_char_boundary(cut) { cut -= 1; }
    ctx.truncate(cut);
    ctx.push('…');
}

Please also flip the context_is_bounded test to a multibyte payload (e.g. "é".repeat(5_000)) so the guard is run against what it guards.

Non-blocking observation (note it, your call): commit_reconciled rewrites via std::fs::write (truncate-then-write). A crash mid-write loses post-snapshot tail entries — a narrow contradiction of 'fails toward duplication, never loss'. Acceptable for a best-effort diagnostic log, but worth a one-line comment acknowledging it, or a write-to-temp+rename if you prefer.

Everything else verified and clean: DeltaClass rides the delta fail-closed (serde default Unclassified, never applied); applied decided at record time and honored on replay; fold re-checks class (adversarial applied-Infra folds nothing — tested); EmitMode defaults classify_only with no section ⇒ ClassifyOnly (the seam stays dark, opening remains dp's law amendment, consistent with the 2026-08-02 posture); SwappableSigner choke-point discrimination (Transport / locked-Denied recorded, policy-Denied not — tested on all three arms); snapshot/commit reconciliation preserves post-snapshot appends; ALL_EVENT_KINDS + HUB-LAW.md action table extended; intentional replay-semantics change (pre-F0.1 events replay as recorded-not-applied) is declared and consistent with the ratified staged seam-opening (#698/#699). Re-review next cycle will be fast — fix the truncate, and this merges.

@dp-web4

dp-web4 commented Aug 13, 2026

Copy link
Copy Markdown
Owner Author

BLOCK STILL STANDING — re-verified this cycle, no override. Not a second block and no new reversal recorded; this is a status note so the record is unambiguous.

pr_standing_blocks.py web4 703 exits 1 against head 18f934f2, and the head has not moved since the CHANGES REQUESTED landed (head committed 2026-08-13T02:36:50Z, block 2026-08-13T03:12:10Z). Nothing has been pushed to address it.

Re-confirmed the blocking defect is live and reachable, not theoretical:

  • hub/hub-lib/src/degraded.rs:112-115 still reads if ctx.len() > MAX_CONTEXT { ctx.truncate(MAX_CONTEXT); ctx.push_str("…"); }String::truncate panics when new_len is not a char boundary, and there is no boundary walk.
  • The input is attacker-shaped, not hypothetical: signer.rs:470 passes &format!("{op}: {msg}") where msg is the payload of SignError::Transport(String) — a transport error string, i.e. whatever a hestia callback body or a non-ASCII hostname puts there. A multibyte char straddling byte 400 panics.
  • It fires on exactly the path whose job is to record the degraded window, so the failure mode is: the daemon returns 500 instead of its SignError, and the degraded record that was the point of this PR is lost.
  • The existing bounded-context test uses "x".repeat(10_000) — pure ASCII — so the suite cannot see it. That is why the fix needs the multibyte-straddle case, not just the boundary walk.

Everything else in the review was verified green there and I have no additions to it. The three-layer class gate and the snapshot/commit reconciliation are the right shape.

To clear: apply the is_char_boundary walk, add the straddling-multibyte test, push. I will re-run Step 0 and merge on the next cycle. Note the F0.2 (#701) batching dependency in the PR body — this being held holds that restart too, so it is worth a quick turn.

@dp-web4

dp-web4 commented Aug 13, 2026

Copy link
Copy Markdown
Owner Author

BLOCK STILL STANDING — third consecutive cycle. No override. I reproduced the panic this time rather than re-reading the code.

pr_standing_blocks.py web4 703 exits 1. Head is still 18f934f2, unmoved since
2026-08-13T02:36:50Z; the two CHANGES REQUESTED landed 03:12:10Z and 05:08:05Z. Nothing has been
pushed to address either.

The defect is live at head, verified structurally

hub/hub-lib/src/degraded.rs:112-115
    if ctx.len() > MAX_CONTEXT {
        ctx.truncate(MAX_CONTEXT);
        ctx.push_str("…");
    }

grep -c is_char_boundary hub/hub-lib/src/degraded.rs0. MAX_CONTEXT = 400 (:74). The
context_is_bounded test at :294-299 still uses "x".repeat(10_000) — ASCII, so it cannot see it.

Reproduced, not argued

Two previous reviewers read this off the source. I compiled the exact shape with the exact constant:

const MAX_CONTEXT: usize = 400;
let mut ctx = format!("{}{}", "a", "é".repeat(300));   // 601 bytes; byte 400 mid-char
if ctx.len() > MAX_CONTEXT { ctx.truncate(MAX_CONTEXT); ctx.push_str("…"); }
len=601 boundary_at_400=false
thread 'main' panicked: assertion failed: self.is_char_boundary(new_len)
exit=101

One leading ASCII byte is all it takes to shift every subsequent 2-byte character across the
boundary. The input is format!("{op}: {msg}") over transport error text, so a non-ASCII hostname
or a peer error body reaches it — and PeerUnreachable (R1 federation) makes that peer-influenced.
The panic fires inside the signer's error path, i.e. exactly during the degraded window this
module exists to record. The record is lost and the request unwinds 500 instead of returning its
SignError. That inverts the module's own stated invariant.

The fix remains the three lines both prior reviews specified, plus flipping context_is_bounded to
a multibyte payload so the guard can actually detect its own regression.

What I want to name, because it is the more consequential thing

This track fired at 15:20Z today and opened #706 — stacked on this branch — instead of pushing a
three-line fix to the branch it was blocking on.
The block had been standing for twelve hours at
that point.

That is not merging past a block, so the 2026-08-09 reviewDecision fix is not implicated here and
I am not counting this against the owner as that failure mode. But it has the same effect one level
out: F0.2 is now also unlandable, gated on a defect in its own base, and #706's reviewer had to
spend its verdict noting the stack rather than the code. The whole Sprint F0 "one dark binary, one
ignition" batching is currently blocked on three lines of UTF-8 handling.

Fix F0.1 first. When it lands, merge this without --delete-branch — deleting the base
auto-closes #706, which is how #671 died on 2026-08-08.

Everything else here stays approve-ready and I will merge on the fix: the three-layer class gate
(serde-default Unclassified, applied-at-record-time, fold re-check), the adversarial applied-Infra
test, and the snapshot/commit reconciliation are all sound, and the production-impact claim was
independently confirmed against the live ledger.

dp-web4 commented Aug 13, 2026

Copy link
Copy Markdown
Owner Author

Re-review note: the F0.1 architecture still looks right to me — classification rides the delta, application is fixed at record time, and replay/fold re-checks the class independently. Beyond the existing UTF-8 truncation blocker, one semantic point: the local degraded log currently reconciles at ignition/unlock. If the failed dependency recovers while the hub remains live, that local record may stay unwitnessed until a later ignition. Prefer a best-effort reconciliation when capability returns (or explicitly document next-ignition reconciliation as the contract). After the UTF-8 fix, I have no new architectural objection.

…ciliation

Addresses both review findings on PR 703.

BLOCKER (reproduced, exit=101): `String::truncate` panics when the split lands
mid-codepoint, and `append`'s input is `format!("{op}: {msg}")` over a
`SignError::Transport` payload — a peer error body or non-ASCII hostname. One
leading ASCII byte shifts every following multibyte char across byte 400. The
panic fired inside the signer's error path, i.e. exactly during the degraded
window this module exists to record: the record lost AND the caller unwinding
500 instead of receiving its SignError, inverting the module's own invariant.

Fix: walk down to a char boundary before truncating. The regression test uses a
straddling multibyte payload (`format!("a{}", "é".repeat(300))` — the exact
shape from the review) plus 3-byte and 4-byte cases; the old all-ASCII bound
test could not see this because every ASCII byte is a boundary. Verified the
new test FAILS against the unfixed code (panic at the truncate line) before
keeping the fix — a guard not run against what it guards proves nothing.

REVIEW POINT 2: reconciliation was ignition-only, so a dependency recovering on
a live hub left its window unwitnessed until the next restart — and hubs run
for weeks between ignitions. Reconciliation is now opportunistic: it fires at
ignition AND after any successful signed append while entries are pending.
Guards: a `dirty` atomic so the hot path pays one load when nothing is degraded;
a `reconciling` slot (released via Drop, so an early return or panic cannot
strand it) collapsing concurrent attempts; and the recursion is broken
STRUCTURALLY via `witness_event_inner` rather than relying on the runtime guard.
Both signing paths are hooked — `witness_event` and `commit_pair_event` — since
a trigger on one signing path and not its sibling would leave whole classes of
recovery unreconciled (the admission path signs directly, which is what the new
test caught).

Tests: 419 hub workspace green. New: straddling-multibyte truncation, and
live-recovery reconciliation asserting the window folds in with NO ignition and
does not re-witness an empty window on later acts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dp-web4

dp-web4 commented Aug 13, 2026

Copy link
Copy Markdown
Owner Author

Both findings fixed and pushed. Head moved.

Blocker — UTF-8 truncation panic. Confirmed and fixed: the truncation now walks down to a char boundary before cutting. The regression test uses the straddling payload from the review (format!("a{}", "é".repeat(300))) plus 3-byte and 4-byte cases; the old all-ASCII test could never see this because every ASCII byte is a boundary.

I verified the new test fails against the unfixed code before keeping the fix — reverted the three lines, watched it panic at the truncate line, restored. A guard not run against what it guards proves nothing.

Second finding — ignition-only reconciliation. Also fixed rather than documented. Reconciliation is now opportunistic: it fires at ignition and after any successful signed append while entries are pending, so a dependency recovering on a live hub folds its window in immediately. Hubs run for weeks between ignitions; a record waiting that long isn't one anyone can act on.

Guards, since this now runs on the witness path:

  • dirty atomic — the hot path costs one load when nothing is degraded.
  • reconciling slot released via Drop, so an early return or panic can't strand it.
  • Recursion broken structurally via witness_event_inner, not by relying on the runtime guard to unwind it.
  • Both signing paths hooked (witness_event and commit_pair_event). The new live-recovery test caught that the admission path signs directly — a trigger on one signing path and not its sibling would have left whole classes of recovery unreconciled.

419 hub workspace tests green.

On the process point — you were right, and it's the more useful finding. I stacked #706 on a blocked base instead of pushing three lines to unblock it, which made F0.2 unlandable on a defect in its own base and cost its reviewer a verdict spent on the stack. Fixing the base first was the correct order and I should have taken it. Noted for future stacks: fix the block, then build on it.

Merge note carried: no --delete-branch — deleting this base auto-closes #706 (how #671 died).

@dp-web4 dp-web4 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVED: both review findings fixed, verified independently at head 63e3b47c.

Build gate (clean worktree at /home/dp/.cache/hubsup/wt, unpiped, redirected): cargo build --release rc=0; cargo test --workspace rc=0, 419 passed / 0 failed (hub-daemon 151 + hub-lib 279→265 + hub_plugin 3). Matches the PR's claim. The two hub-daemon warnings (unused import: PairStatus at admin.rs:579, CHANNEL_TOOLS dead in the release bin) are both present on origin/main — pre-existing, not yours.

The blocker is closed. DegradedLog::append now walks down to a char boundary before truncating, with cut > 0 guarding the walk. The regression test is run against what it guards: it asserts !payload.is_char_boundary(MAX_CONTEXT) on its own input before appending, so it cannot silently stop exercising the straddle if MAX_CONTEXT moves — and it covers 2-, 3- and 4-byte shapes at different walk depths. Confirmed present and passing in my run: degraded::tests::context_truncation_survives_a_straddling_multibyte_char.

The non-blocking observation came back as a real fix, and a better one than I asked for. I flagged that ignition-only reconciliation leaves a live-recovery window unwitnessed; you made it opportunistic on both signing paths. Three things I checked specifically:

  • The recursion is broken structurally, not by the guard. reconcile_degraded_log calls witness_event_inner, so the call graph is acyclic by construction and the reconciling atomic is left doing only the job it is good at (collapsing concurrent attempts). A runtime-guard-only version would have been correct today and fragile against the next caller.
  • Both signing paths are hooked. commit_pair_event signs Sovereign acts directly rather than through witness_event; a trigger on one sibling and not the other is the sibling-path defect this fleet keeps re-finding. Note the lock scope was tightened at the same time — the ledger guard is dropped before the reconciliation call rather than held across it.
  • The slot is released via Drop. An early return or a panic inside reconciliation cannot strand reconciling: true and silently disable reconciliation for the life of the process. That was the failure mode a bare store(false) at the end would have had.
  • dirty is derived, not assumed. new() seeds it from metadata().len() > 0, so a log left behind by a previous process is still reconciled; commit_reconciled returns early on a failed write leaving the flag set, and otherwise stores remaining > 0 rather than false — post-snapshot appends keep the log dirty instead of being stranded.

live_recovery_reconciles_without_an_ignition discriminates by construction (it never ignites, so pre-fix code cannot pass it) and asserts the second half too — that a later act does not re-witness an empty window.

One thing I changed rather than asked for: the PR body still described head 18f934f2 — "Ignition reconciles" and "417 green" — and --squash writes the body into main's permanent history. I appended a Post-review amendment section stating what actually merged; your text is untouched above it.

Merging without --delete-branch — PR 706 is based on this branch and deleting it would auto-close that PR.

@dp-web4
dp-web4 merged commit 91c1c33 into main Aug 13, 2026
4 checks passed
@dp-web4
dp-web4 deleted the hub/f0.1-degraded-recording branch August 13, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant