Skip to content

Truthful tier detection: real headless/subagent/auto signals - #15

Merged
davidcrowe merged 1 commit into
mainfrom
fix/truthful-tier-detection
Aug 16, 2026
Merged

Truthful tier detection: real headless/subagent/auto signals#15
davidcrowe merged 1 commit into
mainfrom
fix/truthful-tier-detection

Conversation

@davidcrowe

Copy link
Copy Markdown
Collaborator

Client half of gatewaystack-connect#692 (see gatewaystack-connect/docs/tier-semantics-auto-vs-interactive.md, §4 and rec 1). Server half: davidcrowe/gatewaystack-connect PR for fix/truthful-tier-labels.

What was wrong

resolveAgentTier() guessed the tier from permission_mode and misreported every case that matters:

Session shape Before After Signal
Interactive terminal/IDE, default mode interactive interactive entrypoint cli / IDE, no unattended signal
Auto mode (human chose it, at the terminal) subagent interactive permission_mode alone never implies unattended
Headless claude -p / cron interactive background CLAUDE_CODE_ENTRYPOINT=sdk-cli
SDK-driven / mcp serve / GitHub Action interactive background sdk-ts / sdk-py / mcp / claude-code-github-action, or CI=true
Task-spawned subagent interactive (undetectable) subagent agent_id on the hook input
Subagent inside a headless run interactive background most-restrictive-wins along the chain
bypassPermissions background background unchanged
Tier detection throws n/a interactive + LOUD lapse.log line + detectError in tier_signals — never brick a client on detection uncertainty

Also in this PR

  • tier_signals (entrypoint, agentId, ci, detectError) ride along on both hook bodies so the gateway can audit what drove the claim.
  • The gateway's new tier-divergence notice (flag, not deny) is relayed as a systemMessage, deduped once per session like the uncredentialed banner. Allow-path exits funnel through one writer so the hook still emits exactly one stdout JSON object.
  • retry-before-fail-closed.test.mjs induced "unattended" via the old auto→subagent mapping and spread the runner's env (CI=true on Actions would have flipped every interactive case to background); it now sets entrypoints explicitly.
  • Version 0.10.3; also fixes the pre-existing plugin.json (0.10.2) vs marketplace.json (0.10.0) drift.

Behavior change worth reviewing

Headless/CI runs now carry an unattended tier, so per #385 they fail closed on gateway outages where they previously failed open as fake-interactive sessions. That is the designed posture finally reaching the sessions it was designed for — but it is a real change for anyone running claude -p under a wildcard key.

Tests

node --test: 61 pass (12 new in test/tier-detection.test.mjs — wire-level, spawning the real hook against a capture server: headless detection, subagent detection, auto-mode regression, CI env, signals payload, notice relay + dedupe, single-JSON-object discipline, deny passthrough).

… permission_mode proxy

The permission_mode heuristic misreported every case that matters
(gatewaystack-connect#692, docs/tier-semantics-auto-vs-interactive.md):
headless/cron `claude -p` runs claimed "interactive" — the loosest tier
for the most unattended shape — Task-spawned subagents were undetectable
(they inherit permission_mode), and auto mode, a human at the terminal,
claimed "subagent".

Detection now reads real signals, most-restrictive-wins:
- bypassPermissions -> background (unchanged)
- headless/programmatic entrypoints (CLAUDE_CODE_ENTRYPOINT sdk-cli /
  sdk-ts / sdk-py / mcp / claude-code-github-action) or CI env ->
  background
- Task-spawned subagent (agent_id on the hook input) -> subagent, unless
  the session itself is unattended
- everything else, including auto mode -> interactive

Detection failure never bricks a call: fall back to "interactive"
loudly — lapse.log line plus a detectError flag in the new tier_signals
body field the gateway records for audit.

The gateway now flags client-reported vs enforced tier divergence with a
non-blocking notice; relay it once per session as a systemMessage
(deduped like the uncredentialed-session banner).

retry-before-fail-closed.test.mjs induced "unattended" via the old
auto->subagent mapping and spread the runner's env (CI=true would flip
every case to background on Actions); it now sets entrypoints explicitly.

Consequence worth naming: headless runs now fail CLOSED on gateway
outages (the #385 posture for unattended tiers) where they previously
failed open as fake-interactive sessions.

Refs gatewaystack-connect#692
@davidcrowe

Copy link
Copy Markdown
Collaborator Author

Fail-closed-on-outage posture for background-labeled sessions confirmed by David 2026-08-16 ("fail closed is the right way to go" — uptime record on GCP acceptable risk). This is a deliberate refinement of the July fail-open-default policy: attended tiers keep fail-open+loud; unattended fails closed and must leave an audited lapse/coverage-gap record, never silent zero rows. Safe to merge on this point.

@davidcrowe
davidcrowe merged commit eb371fb into main Aug 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant