Skip to content

fix(web-security): complete the install without reaching the network - #130

Merged
GangGreenTemperTatum merged 2 commits into
mainfrom
nick/wp1-59-offline-install-guards
Aug 27, 2026
Merged

fix(web-security): complete the install without reaching the network#130
GangGreenTemperTatum merged 2 commits into
mainfrom
nick/wp1-59-offline-install-guards

Conversation

@monoxgas

Copy link
Copy Markdown
Contributor

Companion to dreadnode-tiger#2320 (WP1-59). That PR makes the platform install capabilities without egress; this one makes web-security actually succeed under it.

The problem

install_tools.sh runs on every sandbox boot where the capability changed, not just the first, and every download was unguarded. A runtime that already carried the tooling fetched it all again — and on a disconnected install the first fetch failed, set -e aborted, and the whole provision died before installing anything.

Measured against a runtime image with the tooling pre-baked, --network none:

curl: (6) Could not resolve host: go.dev
tar: Error is not recoverable: exiting now
SCRIPT EXIT=2

Line 15. It never reached anything else.

What changed

Every fetch guarded on the artefact it produces; every version pinned.

  • Five go install ...@latest pinned (pdtm v0.1.5, protoscope, interactsh v1.3.1, 2fa v1.2.0, surf v0.0.5) and guarded. @latest re-resolves against the module proxy even when the binary is present, and yields a different tool set on different days — which no SBOM can describe.
  • pdtm -install narrowed to the tools actually missing instead of re-fetching all nine.
  • katana, agent-browser, its browser binaries, and the caido-mode npm deps guarded the same way.
  • The Go toolchain is now fetched only when something still needs building. It is a ~150 MB download whose sole purpose is building the tools above, and it was the line the entire script died on.

Three latent bugs this surfaced

Once execution got past line 15, these aborted the run on any non-root runtime, connected or not:

  • mkdir -p /opt/burp and the /usr/local/bin/burp wrapper assumed root. Now escalate via as_root and degrade to a warning. The wrapper is only written when the jar actually arrived — a burp on PATH pointing at nothing is worse than no burp.
  • apt-get install for exiftool and Node.js assumed root. Same treatment.
  • pip install assumed a pip binary exists. A uv-managed virtualenv has none, so these died with "command not found" on exactly the images the SDK ships. py_install prefers uv pip, falls back to pip, then python3 -m pip.

Burp, Caido, jxscout and the browser binaries aren't redistributable, so a disconnected deployment won't have them. They now degrade to warnings instead of taking the rest of the tooling down.

The caido-cli check was a false negative

It asserted a local server binary the capability never uses — mcp/caido.py talks to Caido over CAIDO_URL with the Python client. An enclave running its own Caido was reported degraded while working perfectly. Now satisfied by a local binary or a reachable endpoint. Verified all three ways:

no caido, no endpoint     → FAIL  (correct)
reachable at default URL  → PASS  (correct — this was the false negative)
explicit CAIDO_URL        → PASS  (correct)

Result

Same image, same --network none:

WARN: Caido CLI install failed, skipping
WARN: caido-mcp-server download failed, skipping
WARN: Burp Suite download failed, skipping
WARN: jxscout-pro-v2 not found. Set JXSCOUT_BINARY_URL to install...
web-security tools installed successfully
SCRIPT EXIT=0

13 checks passing. The 11 still failing are tools the runtime image doesn't pre-bake yet — the tiger-side manifest tracks the pinned submodule, and main has grown caido-mcp-server, waymore, pacu, fireprox, archivealchemist, exiftool and ast-grep since. Bumping that pin will trip the drift guard in #2320, which is exactly what it's for; pre-baking the additions is the follow-up that takes this to fully green.

Test plan

  • just validate — 26 validated, 0 failed
  • web-security suite — 615 pass (3 failures are pre-existing: datetime.UTC on Python 3.10, confirmed failing on a clean tree)
  • pre-commit clean
  • New test_install_tools_offline.py pins every-fetch-guarded and every-version-pinned — all 7 fail against the previous script
  • End-to-end in a pre-baked runtime image on --network none: exit 0

Version bumped 1.13.01.14.0 — install behaviour and pinned tool versions change, though no public surface does.

The install script runs on every sandbox boot where the capability changed,
not just the first. Every download was unguarded, so a runtime that already
carried the tooling fetched it again anyway — and on a disconnected install
the first fetch failed, `set -e` aborted, and the entire provision died at
line 15 before installing anything. Measured against a runtime image with the
tooling pre-baked: `SCRIPT EXIT=2`, nothing installed.

Guard every fetch on the artefact it produces, and pin every version:

- `go install ...@latest` x5 pinned (pdtm v0.1.5, protoscope, interactsh
  v1.3.1, 2fa v1.2.0, surf v0.0.5) and guarded. `@latest` re-resolves against
  the module proxy even when the binary is present, and produces a different
  tool set on different days — which no SBOM can describe.
- `pdtm -install` now installs only the tools actually missing, rather than
  re-fetching all nine every run.
- katana, agent-browser, its browser binaries, and the caido-mode npm deps
  guarded the same way.
- The Go toolchain is a ~150 MB download whose only purpose is building the
  tools above, so it is now requested only when one of them is missing. That
  was the line the whole script died on.

Three latent bugs surfaced once execution got that far, all of which aborted
the run on a non-root runtime regardless of connectivity:

- `mkdir -p /opt/burp` and the `/usr/local/bin/burp` wrapper assumed root.
  Now escalate through `as_root` and degrade to a warning. The wrapper is only
  written when the jar actually arrived — a `burp` on PATH pointing at nothing
  is worse than no `burp`.
- `apt-get install` for exiftool and Node.js assumed root, same treatment.
- `pip install` assumed a `pip` binary. A uv-managed virtualenv has none, so
  these died with "command not found" on exactly the images the SDK ships.
  `py_install` prefers `uv pip`, falls back to pip, then `python3 -m pip`.

Burp, Caido, jxscout and the browser binaries are not redistributable, so a
disconnected deployment will not have them. Those now degrade to warnings
rather than taking the rest of the tooling down with them.

Also: the `caido-cli` check asserted a local server binary the capability
never uses — it talks to Caido over `CAIDO_URL` with the Python client
(mcp/caido.py). An enclave running its own Caido was reported degraded while
working correctly. Now satisfied by a local binary *or* a reachable endpoint;
verified all three ways.

Verified in a runtime image with the tooling pre-baked, `--network none`:
`SCRIPT EXIT=0`, 13 checks passing, every unavailable vendor tool warning
rather than aborting.

New tests pin both properties — every fetch guarded, every version pinned —
and all seven fail against the previous script.
…ine guard tests. Guard py_install/curl/git-clone fetches, enforce as_root on every root-owned path write, gate go clean on need_go, and cover all 14 WARN fallback markers.

@GangGreenTemperTatum GangGreenTemperTatum left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added as_root escalation to every /usr/local/bin write that was missing it (Caido CLI, caido-mcp-server, kiterunner, jxscout, npm install -g agent-browser) — without this, any of them abort the entire provision under set -e on a non-root runtime, same failure mode the Burp fix already addressed. Also gated go clean on need_go so it skips when Go was never invoked, switched kiterunner to use have kr for consistency, and expanded the structural test suite from 8 to 20 tests covering py_install/curl/git-clone guards, as_root enforcement on all root-owned path writes, and all 14 WARN fallback markers.

@GangGreenTemperTatum
GangGreenTemperTatum merged commit c2a8de4 into main Aug 27, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants