feat(harness): agent-sdk adapter with per-call tool gate (#21) - #92
Conversation
Adds the seam the agent-sdk adapter and the tool gate build against: HarnessToolGate and its decision types, callGateFailClosed (a throw or a malformed return is a deny), HarnessInvocation.gate, the optional canGatePerCall capability with its passthroughs, and a gate-decision harness event. createHarnessToolGate is a deny-everything placeholder. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The reason is journaled, and the thrown message was passed through uncapped and with control characters intact. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…l gate (#21) Adds the agent-sdk adapter: query() with spawnClaudeCodeProcess so the CLI runs detached in its own cgroup, killed through killContained on timeout, idle timeout and hold. PreToolUse calls the invocation gate through callGateFailClosed and emits a gate-decision event for every call. A hold denies the call, ends the process and throws HARNESS_GATE_HOLD_CODE. Reuses the claude usage parsing and rate-limit classification and the #70 event mapper, now exposed for already-parsed messages. Registered as agent-sdk; it passes the containment conformance suite. Adds @anthropic-ai/claude-agent-sdk 0.3.284. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…stations (#21) An adapter with canGatePerCall receives a gate on every maker and critic invoke. A throw carrying HARNESS_GATE_HOLD_CODE moves the card to the hold lane without spending an execution attempt; its recovered usage goes through the existing maker-throw and critic fold sites, so no fold site is added. The critic gate may write only verdict.json. The new test file guards the maker-throw and gate-critic folds in the mutation manifest. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Adds the gate-decision row kind to harness_events: decision, code, subagent id and a reason stripped of control characters and cut to 200 characters. The journal DB has no version number, so the columns arrive through its idempotent ALTER ladder. journal inspect and tail print the rows, including for an invocation with no span. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Runs behind CONDUIT_E2E_AGENT_SDK=1: real usage and cost through an allow-all gate, and a Bash write blocked by a deny gate with the deny decision emitted. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…the exit wait (#21) The executor passed ownedPaths: [] to the gate for a card with no owned paths in an enforcing flow. runOwnedPathsIntegrity treats that as an opt-out, so the gate would have denied every write the integrity check leaves unenforced. Pass ownedPaths only when the card declares some. The adapter's wait for the CLI exit event left its 5 s timer running, holding the process open after every call. Clear it. Adds executor tests against the real gate: owned paths enforced, not enforced without the flow opt-in or without card paths, and the critic gate allows only verdict.json. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…l gate (#21) Adds the middle containment claim to harness-containment.md, the agent-sdk registration notes, and the per-call gate entry in the CLAUDE.md wired inventory. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (24)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Nitpick review — comment
The change adds a new agent-SDK harness adapter with per-invocation cgroup containment, gate-hold plumbing in the executor, and gate-decision journaling, and it is largely well-tested. Three real defects survive: the adapter tracks only the most recently spawned child, so any second spawn escapes killAll and the finally-block cleanup; the gate-hold path in executeHarnessStation passes err where the catch binding is invokeErr, mis-attributing (or breaking) the hold; and the run-scoped config dir leaks if the SDK import or containment probe throws before the try/finally begins. The CLAUDE.md journal-kind inventory is also now stale (missing gate-decision), and the rework-pass critic-hold branch lacks a test. Verdict: comment — fix the tracking slot, the err/invokeErr mixup, and the config-dir cleanup window before merge.
5 inline comment(s).
…optional deps in the image (#21) An adapter that gates per call denies every tool the station does not list, so an empty tools list or unrestricted_tools: true leaves the station unable to write its own output. Fail at load with HARNESS_GATED_ADAPTER_NEEDS_TOOLS instead of at first dispatch. The image installs with --omit=optional: the Agent SDK's optional platform binaries are about 460 MB and the agent-sdk adapter runs the claude on PATH. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A hold now answers the call with a deny and continue:false, so the CLI stops and emits its result message, and the thrown error carries the call's usage and cost. Every later PreToolUse call is denied without asking the gate. The wait is bounded by HOLD_STOP_WAIT_MS (5 s), after which the process group and cgroup are killed as before. The wall-clock and idle timers keep running during the wait, and the hold still wins over timeouts and errors. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change introduces an agent-sdk harness adapter with per-call tool gating, gate-decision journaling threaded through persistence and the executor, and solid test coverage. Two issues warrant fixing before merge: the adapter tracks only the most recently spawned child process, so any additional spawn leaks its process group and cgroup past the containment boundary, and the maker-path gate-hold in the executor passes err to escalateToHold where the catch binding is invokeErr, mis-attributing (or failing to compile) the hold's error. Remaining items are non-blocking: a stale 'writes only' kind list in CLAUDE.md that now contradicts the gate-decision sentence in the same paragraph, raw stderr tails embedded in thrown error messages, a misleading resolution failure for relative executable paths, a scrap-shaped journal outcome on held cards, and potentially flaky wall-clock ceilings in the new tests.
7 inline comment(s).
There was a problem hiding this comment.
Nitpick review — comment
The change adds a gated agent-sdk harness adapter with cgroup containment, gate-decision journaling, and load-time validation, well covered by tests. Two real risks survive: the adapter tracks only the most recent spawned child, so any second CLI process would escape cleanup, and the maker gate-hold path passes the wrong error binding (err instead of invokeErr) to escalateToHold. The change's breadth — a new process-execution path, SQL writes, and deploy config — warrants human review before merge.
5 inline comment(s).
- Fix: track every child the SDK spawns, so killAll and cleanup cover all of them - Fix: start the config-dir try/finally right after creation so an options-build throw cannot leak it - Fix: distinguish a relative executable path from a missing binary in probe and invoke errors - Fix: list gate-decision among the journal kinds in CLAUDE.md - Test: reject, back-edge, held-critic on a rework pass spends no attempt - Test: loosen wall-clock ceilings that could flake on loaded CI Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Main carries the squashed #91. Conflicts in harness-gate.ts and its test resolved to main's version, which canonicalizes owned paths once at build time. harness-events-journal.ts keeps this branch's gate-decision persistence. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
This PR integrates a new Agent SDK harness adapter with per-call tool gating, gate-hold escalation in the executor, gate-decision journaling, and Docker packaging. The core design is sound and well-tested, but the maker-path gate-hold escalation passes the wrong error variable (err instead of invokeErr), so the hold's journal attribution carries the wrong error object or fails outright. Given the new execution surface and infra changes, human review of the change is recommended before merge.
3 inline comment(s).
1 previously-acknowledged finding(s) not re-posted (resolved review threads).
There was a problem hiding this comment.
Nitpick review — comment
The change adds an Agent SDK harness adapter with per-call tool gating, a new 'hold' gate-decision path through the executor, and the supporting persistence and Docker changes. Two real risks survive: the gate-hold call in executeHarnessStation passes err where the surrounding scope binds the thrown error as invokeErr, which either fails to compile or attributes the hold to the wrong error; and the Dockerfile drops the SDK's pinned binaries in favor of whatever claude resolves to on PATH, an unpinned executable running with the station's tool permissions. The journal's reason sanitization invariant also rests entirely on the producer side with no enforcement or test at the write boundary. Given the execution-path and infra exposure, human review is recommended; verdict is comment — no blocking defect, but the err/invokeErr mismatch and the unpinned binary should be fixed before merge.
4 inline comment(s).
1 previously-acknowledged finding(s) not re-posted (resolved review threads).
- Fix: split the gated-adapter load error so a waiver with a tools list no longer claims the station can do nothing - Test: pin both message cases - Docs: state that only claude-headless runs named agents Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change adds a substantial agent-SDK harness adapter with per-call tool gating, event journaling, and Docker packaging, and the core gating and journaling paths look coherent. Two real issues survive review: the gate-hold escalation in the executor passes err where the surrounding code consistently uses the caught invokeErr, risking the wrong error being journaled on the hold path, and the new harness_events schema comment promises reason sanitization that the persistence layer's own insert path does not perform. Given the breadth of exec exposure, SQL changes, and infra edits in this diff, human review is recommended before merge.
3 inline comment(s).
3 previously-acknowledged finding(s) not re-posted (resolved review threads).
- Fix: harness_events schema comment names the writer as the sanitizer - Test: isDead treats only ESRCH as dead, so EPERM cannot pass a kill assertion - Test: the agent-sdk E2E no longer pins usage.model to haiku Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change introduces an agent-SDK harness adapter with per-call tool gating, gate-hold escalation, and harness-event journaling, and it is well covered by dedicated tests. Two issues warrant attention: the maker-path gate-hold escalation passes err where the caught binding in scope is invokeErr (a likely ReferenceError or mis-attributed hold cause), and the new journal migration ladder for pre-existing harness_events tables has no test exercising the ALTER path against an old-schema database. The agent-sdk adapter also documents that named agents are unimplemented but does not fail closed if one is configured. None of these are merge-blocking, but the err/invokeErr mismatch should be fixed before merge.
1 inline comment(s).
2 previously-acknowledged finding(s) not re-posted (resolved review threads).
- Test: the harness_events migration test asserts every added column reads null on a legacy row and round-trips on a new gate-decision row Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change wires a new agent-sdk harness adapter (with per-call tool gating, run-scoped config isolation, and gate-decision journaling) into the executor, persistence, and docs, and is well covered by a large new test suite. Two things warrant attention before merge: the run-scoped config directory is created just outside the try whose finally cleans it up, so a throw from env building would leak a directory containing a credentials symlink, and the new hold path passes err to escalateToHold while interpolating invokeErr — worth confirming they are the same object, since that error is what the human resolving the hold sees. Verdict: comment; no blocking issues.
2 inline comment(s).
2 previously-acknowledged finding(s) not re-posted (resolved review threads).
- Fix: build the child env before the run-scoped config dir exists, so no step between creation and the try can leak it - Test: loosen the hold-wait lower bound from 180 ms to 150 ms Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change wires the new agent-sdk harness adapter, per-call gating, and harness event journaling into the executor and CLI, and the overall structure matches the existing harness patterns. One real defect survives: the maker gate-hold path passes err to escalateToHold inside a catch that binds invokeErr, so the hold record can carry the wrong or a missing error. Two lower-severity items — an empty critic tool allowlist that can wedge gates into permanent holds, and missing coverage for the new isolate-config wiring — are worth addressing but non-blocking.
2 inline comment(s).
1 previously-acknowledged finding(s) not re-posted (resolved review threads).
- Test: the registry accepts _ISOLATE_CONFIG on agent-sdk and still rejects _AGENT and _PLUGIN_DIRS there Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change adds the agent-SDK harness adapter with per-call tool gating, the gate-hold path through the executor, and supporting journal/loader/docs work, and it is largely coherent with the existing gate and budget machinery. One real defect survives: the hold escalation in executeHarnessStation passes err where every other reference in the block uses invokeErr, so the hold's recorded cause may be the wrong error object. Two smaller items: child stderr tails flow unfiltered into persistent error/journal metadata, and the waived-case validation message steers users into a sibling rejection.
1 inline comment(s).
2 previously-acknowledged finding(s) not re-posted (resolved review threads).
The it.each table was typed as const, which made pluginDirs readonly and not assignable to HarnessAdapterConfigDef. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change adds a new Agent SDK harness adapter with per-call tool gating, a new 'hold' GateDecision path, journal persistence for gate decisions, and a slimmer production Docker image. The core design is sound and well tested, but three issues warrant attention before merge: the maker gate-hold path passes the wrong error binding to escalateToHold, the new adapter silently bypasses the documented named-agent hold-on-miss contract, and the hold reason can degrade to an empty string on an empty error message. Given the change's blast radius across execution, gating, and journaling, human review is recommended.
3 inline comment(s).
4 previously-acknowledged finding(s) not re-posted (resolved review threads).
- Fix: a held critic whose error message is empty falls back to String(err) for its hold reason - Fix: the waived-with-tools load error no longer offers dropping the list, which the sibling check rejects - Test: a held critic with an empty error message still gets a readable terminal reason Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change adds an Agent SDK harness adapter wired through the controller, worker, gate, and persistence layers, with substantial accompanying tests. Two things warrant attention before merge: the gate-hold path in the executor passes err to escalateToHold while the caught error in that scope is invokeErr, which either throws or attributes the hold to the wrong error; and a held card is journaled with a scrap-shaped outcome, which can misclassify it for consumers that read span outcomes. Smaller items: the gate-hold reason reaches persisted surfaces without the sanitization the journal writer applies, and a docs paragraph was inserted between a sentence and the code block it introduces.
1 inline comment(s).
3 previously-acknowledged finding(s) not re-posted (resolved review threads).
- Docs: put the named-agents note before the sentence that introduces the plugin-dir example, so the colon leads into its code block Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change adds an Agent SDK harness adapter with per-call gating and a new gate-hold escalation path in the executor, with substantial test coverage. Two real issues survive: the gate-hold escalation passes err where the catch binding is invokeErr, so the hold record can carry the wrong error (or fail to compile), and the journaled span outcome conflates a held card with a scrapped one. A minor diagnostics issue in the adapter's exit-code tracking is also worth a look; nothing here is merge-blocking.
2 inline comment(s).
2 previously-acknowledged finding(s) not re-posted (resolved review threads).
- Fix: a nonzero exit code from any spawned child is reported over an earlier clean or unknown one Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change introduces a new agent-sdk harness adapter with strong accompanying test coverage, plus gate/executor wiring and a Dockerfile tweak. One real defect survives: the maker-path hold escalation in executor.ts passes err where the caught error is invokeErr, so the hold can journal the wrong cause. A few lower-severity items (an abort-listener edge case in the adapter's spawn path, a potentially flaky wall-clock assertion, and untested env-option routing for the new adapter) are worth addressing; given the change's size and its reach into exec and persistence paths, human review is recommended before merge.
2 inline comment(s).
2 previously-acknowledged finding(s) not re-posted (resolved review threads).
- Fix: end a child at once when its spawn signal has already aborted, since the abort listener never fires on it - Test: a child spawned with an aborted signal is dead while the stream is still open - Test: widen the hold-wait ceiling to 20 s against a 60 s bound Addresses review comments from github-actions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Nitpick review — comment
The change introduces a new agent-sdk harness adapter with per-call tool gating, a gate-hold decision path, and config wiring, with substantial accompanying tests. Three issues need attention before merge: the maker gate-hold path in executor.ts passes err where the catch-bound invokeErr is clearly intended, the Dockerfile now relies on an unpinned claude binary resolved from PATH at runtime with no install step in the image, and the new hold GateDecision branch makes the documented five-fold-site inventory (and possibly the mutation-check manifest) stale, with a risk that the hold path's critic usage is never folded into budgets. Given the wide blast radius of this change to the worker/executor core, human review is recommended.
2 inline comment(s).
4 previously-acknowledged finding(s) not re-posted (resolved review threads).
Part of #21 (slice 3). Adds the
agent-sdkharness adapter, which drives the Claude Code CLI through@anthropic-ai/claude-agent-sdkquery()and asks the kernel's tool gate before every tool call. It is stacked on #91 (the gate and its contract). Its base isfeat/21-tool-gateso the diff shows only this slice, and GitHub retargets it tomainwhen #91 merges.What is in it
src/worker/harness-adapter-agent-sdk.ts), registered asagent-sdkwithcanGatePerCall. It relies on what the Supervised harness adapters: per-call gating via vendor SDK callbacks (Claude Agent SDK, Codex app-server, OpenCode SDK) #21 spikes measured:spawnClaudeCodeProcessstarts the CLI detached, in its own per-invocation cgroup, registered withtrackProcessGroup, and ended with the samekillContainedthe harness runner uses. This reaches asetsidgrandchild (Harness containment does not reap Claude Bash-tool commands: the CLI setsids them out of the process group #77).process.env. It defaults to theclaudeonPATH, resolved to a file, not the SDK's bundled binary.hooks.PreToolUsecalls the gate throughcallGateFailClosed. An allow returns nopermissionDecision, soallowedToolsstill decides and nothing is widened.harness-timeout,harness-idle-timeout,harness-rate-limited(parks as before) andharness-nonzero-exit. Authentication failure is classified onassistant.errororterminal_reason, never onsubtype, which readssuccessfor it.mapClaudeStreamMessage, split out ofmapClaudeStreamLine), so rate-limit and usage events reachonEventunchanged._ENV,_COMMAND,_MODELand_ISOLATE_CONFIG._AGENTand_PLUGIN_DIRSfail registry construction. No session resume.canGatePerCall. Owned paths reach it only when the flow enforces them and the card declares some, the conditionrunOwnedPathsIntegrityuses. The critic gate may write onlyverdict.json. Aholdmoves the card toholdwithout spending an execution attempt, on the maker and critic paths. No fold site was added or moved, and the hold's recovered usage goes through the existing folds. The mutation manifest guards now name the new test file too.gate-decisionis aharness_eventskind withdecision,gate_code,agent_idand a 200-character reason (no tool input bodies).journal inspectandtailprint it, including(no span)invocations. The journal DB has no version number, so the four columns come from its existing idempotentALTERladder.continue: falseand astopReason, so the CLI stops and emits its result message (about 15 ms in a live run, which returned 14453 tokens and $0.03). The thrown error carries that usage, and the existing folds bill it. After the first hold every later call is denied without asking the gate. If no result arrives within 5 s (HOLD_STOP_WAIT_MS) the process is killed as on a timeout, and the error carries no usage.interrupt()was measured and rejected: it yields an error result and the iterator throws. A deny alone was rejected too: the model retried the tool.toolslist orunrestricted_tools: truefails at load withHARNESS_GATED_ADAPTER_NEEDS_TOOLS. Either would deny every tool, including the Write the station needs for its output.docs/harness-containment.md, theagent-sdknotes indocs/harness-adapter-registration.md, and the CLAUDE.md wired inventory.Review fix on top of the slice
The executor passed
ownedPaths: []to the gate for a card with no owned paths in an enforcing flow.runOwnedPathsIntegritytreats that as an opt-out, so the gate would have denied every write the integrity check leaves unenforced. The new executor tests run against the real gate and fail without the fix. The adapter's wait for the CLIexitevent also left a 5 s timer running after each call.Dockerfile
bun addpulls the SDK's optional platform binaries (232 MB glibc, 226 MB musl), which the adapter does not use because it runs theclaudeonPATH. The image now installs with--omit=optional. Checked against the real lockfile: 110 MB instead of 568 MB, and the SDK still imports.Known limits
continue: false. It did in every live run, but the 5 s kill fallback (no usage) was only unit-tested. Parallel tool calls in one turn (the latch denies the later ones), a hold from inside a subagent, and a held critic were not observed live.pathToClaudeCodeExecutable, and a host without cgroup v2 (unit tests inject process-group containment).Tests
bun run test:mutationprotects all 5 fold sites.bun test blackbox/passes 70.tscis clean.setsidgrandchild.CONDUIT_E2E_AGENT_SDK=1) ran twice against haiku: usage and cost populated, and a Bash write was blocked by the gate.🤖 Generated with Claude Code