Make creative sanitization opt-in and restore creative iframe origin isolation

[aram356]

Fixes #955
Stacks on #916 — targets feature/optional-creative-rewriting, extending the same "creative processing is opt-in" idea from rewriting to sanitization.

Summary

Creative sanitization runs unconditionally on every markup bid, stripping script/object/embed/form and friends together with their inner content. For script-based creatives — the majority of programmatic display — that leaves nothing renderable, and the slot goes blank with no error: what survives is usually a tracking pixel, so adm is non-empty and the ad server records a normal impression.

Measured over 291 creative deliveries on a live publisher: a median 43% of bytes removed, 29 creatives reduced by more than 80%, and 20 reduced below 500 bytes. One bidder lost 100% of every creative; another lost 76% across 43 of them.

Changes

1. auction.sanitize_creatives — gates sanitize_creative_html, independent of rewrite_creatives. Lets a publisher disable stripping when creatives render in a foreign-origin frame (the Prebid Universal Creative inside the ad server's iframe — the context rewrite_creative_html already documents), where the markup cannot reach the publisher origin. Opt-in: defaults to false.

2. rewrite_creatives keeps its true default — an earlier revision of this PR flipped it to false; both reviewers asked to keep the base branch's default (the measurement above justifies opt-in sanitization, not disabling rewriting), so only sanitization behavior changes for existing deployments. The new field mirrors the branch's skip_serializing_if pattern: defaults are omitted from blobs, non-default values (sanitize_creatives = true, rewrite_creatives = false) are serialized.

3. allow-same-origin removed from CREATIVE_SANDBOX_TOKENS — in this PR, not a follow-up. The resulting opaque-origin iframe cannot fetch first-party endpoints (CORS, Origin: null), so:

• the click guard recovers mutated clicks via a GET /first-party/proxy-rebuild navigation (302 chain), now registered in every adapter and tolerant of origin-form URIs; URL resolution pins document.baseURI (srcdoc inherits the parent URL; location.href is about:srcdoc), and the GET fallback never overwrites the canonical data-tsclick;
• dynamic resource signing inside creative iframes is disabled pending a parent postMessage broker (Same-origin parent postMessage broker for dynamic signing from opaque-origin creative iframes #982); server-side rewrites are unaffected.

4. Processing invariants hardened — the 1 MiB per-creative cap is enforced in every mode (not just inside the sanitizer); a supplied creative that processing rejects suppresses the PBS Cache fallback (which would otherwise deliver the raw cached adm); and the rewrite pass strips bidder <base> elements itself, since a base href would rebase the emitted root-relative first-party URLs onto a third-party origin.

Why the sandbox change belongs here

Sanitization is documented as "the primary defense against malicious markup", with the sandbox as defense-in-depth. But the sandbox granted allow-same-origin alongside allow-scripts, which removes its origin isolation entirely — the codebase says so itself on ADM_IFRAME_SANDBOX, and APS_RENDERER_SANDBOX omits the token for the same reason. CREATIVE_SANDBOX_TOKENS was the only one granting it.

With sanitization now optional, that pairing would leave creative markup able to reach publisher cookies, storage, and same-origin fetches. The client-side sanitizeCreativeHtml is no backstop — it validates type and emptiness only and returns input unchanged (removedCount is always 0). So the origin boundary moves to isolation, where it no longer depends on an optional transform.

Upgrade and rollback

Binaries that predate sanitize_creatives reject a blob carrying it (deny_unknown_fields), so upgrade the binary first, then push config; the configuration guide documents the sequencing and the unsafe interval. Before rolling back to a binary that predates a field, restore its default and push the default-compatible blob. Environment overlays require both [auction] leaves to exist in the TOML (EdgeZero v0.0.4 cannot create missing leaves).

Verification

• trusted-server-core lib suite 1762 passed, 0 failed; JS suite 432 passed; CLI suite (incl. new sanitize-overlay coverage) green; all six clippy targets and cargo fmt clean; cross-adapter parity tests pass
• Regression tests for: origin-form GET rebuild 302, <base> stripping in both rewrite passes, PBS Cache fallback suppression (script-only / oversized) and retention (absent creative), byte-for-byte pass-through, all four processing modes, oversized rejection per mode, opaque-origin click fallback and two-wave mutation rebuild
• End to end through the dev proxy against a live publisher: sanitize disabled, rewrite disabled with raw == sanitized == output on every creative (8902 → 8902 and 22069 → 22069 for bidders previously losing 100% and 35%), page serving ads with no hydration errors

[prk-Jr]

Summary

The creative-processing behavior change looks coherent: sanitization and rewriting are independently opt-in, and the creative iframe sandbox no longer combines allow-scripts with allow-same-origin. I did not find blocking issues in the runtime/config changes.

Non-blocking

📝 note

• Update stale auction docs/comments for the new creative-processing defaults: crates/trusted-server-core/src/auction/README.md still describes the old contract: creative HTML is always sanitized, rewriting is enabled by default, and [auction].rewrite_creatives = false skips rewriting but not sanitization. That is now stale with this PR: both sanitize_creatives and rewrite_creatives default to false, and sanitization can be disabled independently. Please update the operator-facing README language around crates/trusted-server-core/src/auction/README.md:252 and crates/trusted-server-core/src/auction/README.md:386 to describe the new matrix: default raw bidder adm, sanitize_creatives = true strips executable markup, and rewrite_creatives = true applies proxy/click/runtime rewriting to the current creative HTML.

CI Status

• browser integration tests: PASS
• integration tests: PASS
• integration tests (Fastly EC lifecycle): PASS
• prepare integration artifacts: PASS
• targeted local regression check: PASS (cargo test --package trusted-server-cli --target aarch64-apple-darwin migrated_legacy_config_applies_rewrite_creatives_environment_override)

[ChristianPavilonis]

Summary

I found one confirmed blocker in the opaque-origin creative runtime, along with high-risk upgrade and rollback compatibility problems. The sandbox isolation direction is sound, but rewritten clicks can currently fail, existing configurations silently change behavior, and the documented rollback sequence is now unsafe. Please address the inline findings before merge.

[prk-Jr]

Summary

Adds auction.sanitize_creatives, makes both creative-processing controls opt-in, removes allow-same-origin from CREATIVE_SANDBOX_TOKENS, and recovers the resulting opaque-origin click path via a GET /first-party/proxy-rebuild navigation fallback registered in all four adapters.

The sandbox change is the right call and is correctly scoped: the core handler already answered GET with a 302 (proxy.rs:1899) and validates the original tstoken before re-signing, so the new registrations expose no new capability. Every finding from the previous round is genuinely addressed — the 1 MiB cap is hoisted out of the sanitizer into the shared helper, the (false, true) mode is covered at both levels, the pass-through test asserts a real assert_eq! against the bidder input, and the doc sweep is thorough.

Two things are still blocking, one substantive and one about what the docs promise.

Blocking

🔧 wrench

• rewrite_creatives default flip is unrelated to the bug being fixed (auction_config_types.rs:92): the measurement justifies making sanitization opt-in; flipping rewriting too silently disables first-party proxying, click conversion and creative TSJS injection for every existing deployment, and inverts the base branch's skip_serializing_if rollback contract. Recommend the one-line option the PR description already offers — keep default_rewrite_creatives() -> true.
• Dynamic resource proxying is now permanently off inside creative iframes (proxy_sign.ts:31): signProxyUrl returns null in an opaque origin, dynamic_src_guard.ts:91 falls back to the raw URL, and runtime-injected <img>/<iframe> load direct from third parties. Deferring the postMessage broker is fine; landing it without a tracked issue number and without saying so in docs/guide/creative-processing.md is not — the guide still advertises the injected runtime's click/resource protections unqualified.

Non-blocking

♻️ refactor

• Redundant flag in build_bid_map_omits_oversized_adm (publisher.rs:7564): the cap is unconditional now, so the override only removes coverage of the shipped default mode.
• Duplicated opaque-origin predicate (click.ts:152 vs proxy_sign.ts:27), with divergent catch-path semantics.

🤔 thinking

• The four-mode matrix omits the cap: both guides describe the default (false, false) mode as "deliver the creative exactly as the bidder returned it", but a creative over 1 MiB now yields an empty string and the adm is dropped entirely. The CHANGELOG says "subject to the 1 MiB per-creative cap" — worth the same clause in docs/guide/configuration.md and docs/guide/creative-processing.md, since those tables are what operators read when choosing a mode.

⛏ nitpick

• Two "sanitized, non-executable" claims survived the doc sweep, both in security-relevant comments and neither reachable by an inline comment since the lines are untouched:
  • crates/trusted-server-js/lib/src/core/render.ts:177 — "Construct a sandboxed iframe sized for sanitized, non-executable creative HTML."
  • crates/trusted-server-js/lib/src/core/request.ts:82 — "Render a creative by writing sanitized, non-executable HTML into a sandboxed iframe."
• auction_config_types.rs:30 still documents the field as rewriting sanitized HTML.

👍 praise

• The cap moved to the right layer. Treating the 1 MiB limit as a delivery invariant in process_auction_creative_with_rewriter rather than a sanitizer side effect is the correct fix, and process_auction_creative_rejects_oversized_markup_in_every_mode pins it across all four combinations instead of just the new one.
• The pass-through test proves the actual contract. assert_eq!(adm, original) at both the helper and response-converter level is what makes "ships exactly as the bidder returned it" a testable claim rather than a marker check.
• Opaque-origin recovery is covered end to end — unit tests asserting fetch is never called, plus GET route registration verified per adapter (including a dedicated Spin route test) rather than assumed from the core handler.

Verified non-issues

• </script> breakout from raw bidder markup into the publisher origin: blocked by html_escape_for_script (publisher.rs:3142), which escapes <, >, & before the bids=JSON.parse("…") inline script.
• The GAM direct-replace path already used ADM_IFRAME_SANDBOX without allow-same-origin, so it is unaffected.
• enableDebugFromEnv's localStorage access is already try-wrapped, so the opaque origin's SecurityError won't take down the click guard at install time.

CI Status

• fmt: PASS
• clippy / cargo check (fastly, axum, cloudflare native + wasm, spin native + wasm): PASS
• rust tests (fastly, axum, cloudflare, spin, ts CLI, cross-adapter parity): PASS
• js tests (vitest) + format-typescript + format-docs: PASS
• integration tests (browser, Fastly EC lifecycle, prepare artifacts): PASS

[ChristianPavilonis]

Summary

I found seven additional issues in the current head: one security/control-bypass blocker, four high-severity correctness or compatibility failures, and two medium-severity regressions. In particular, rejected creatives can be restored raw from PBS Cache, and the opaque-origin click recovery is broken independently in the browser runtime and in origin-form request handling. All findings are inline.

[prk-Jr]

Summary

Every finding from the previous round is genuinely addressed. build_bid_map now distinguishes absent / accepted / rejected creatives, the GET rebuild parses uri().query() so origin-form request targets work, <base> stripping moved into the shared rewrite_creative_html_impl (so it is independent of sanitization), TRUSTED_BASE_URL pins resolution to document.baseURI, persistRebuiltClick keeps the canonical data-tsclick out of the fallback's reach, rewrite_creatives is back to its default true, and the #982 deferral is tracked in both the code and the guide. The doc sweep, upgrade sequencing, and CLI overlay coverage are all in place.

One question about a behavior change that goes slightly beyond what was asked, plus three non-blocking observations. Nothing blocking.

Blocking

❓ question

• hb_cache_* is dropped for accepted creatives, not just rejected ones — the cache-fallback fix also removes the coordinates on the normal accepted path, which are GPT slot targeting keys the Prebid Universal Creative reads directly. Likely intended, but undocumented (crates/trusted-server-core/src/publisher.rs:3259, inline).

Non-blocking

🤔 thinking

• The about:srcdoc branch is untested — jsdom never exercises it, and the Playwright/chromium harness the deferral note says is missing already runs in CI (crates/trusted-server-js/lib/src/shared/origin.ts:24, inline).
• #982's blast radius is narrower than the note implies — dynamic signing only installs under renderGuard, which defaults to false (docs/guide/creative-processing.md:89, inline).
• Post-fallback mutation diffing degrades to a warn. Once the opaque-origin fallback is written into href, every later observer pass canonicalizes /first-party/proxy-rebuild?... against the canonical data-tsclick, fails diffParams, and logs click base changed; keeping original at warn level (crates/trusted-server-js/lib/src/integrations/creative/click.ts:236). I traced the loop: it terminates, href is not reverted, and the second wave still rebuilds — so this is correct, not a defect. The residual gap is a creative that mutates by appending to the current href rather than replacing it wholesale; rebuilds a second mutation wave after an opaque-origin fallback replaces the href outright, so that shape is not covered.

⛏ nitpick

• Spin GET route test asserts only != 404 while its comment says "the handler 302s" (crates/trusted-server-adapter-spin/tests/routes.rs:521, inline).

👍 praise

• The Some(adm) / Some("") / None tri-state is the right shape for the reported bypass, and assert_no_render_source pins both halves — script-only under sanitization and oversized under default settings — rather than only the mode that motivated the fix.
• <base> stripping went into rewrite_creative_html_impl, shared by the root-relative /auction pass and the absolute-URL inline pass, with a test at each level. Putting it in the shared impl is what makes "rewriting must not depend on sanitization" structurally true instead of a convention.
• persistRebuiltClick expresses the guard positively — "write data-tsclick only if this is itself a signed /first-party/click" — rather than blacklisting the rebuild URL. That stays correct if another fallback shape is ever added.
• proxy_rebuild_get_with_origin_form_uri_redirects builds a real signed tsclick and asserts the 302 Location plus the applied add/del mutations, instead of the weaker "did not 404" check.

Verified non-issues

• The GET branch validates the original tstoken through reconstruct_and_validate_signed_target before re-signing (proxy.rs:1795), so registering GET in every adapter exposes no open redirect.
• No creative-bundle module other than proxy_sign depends on a usable origin; shared/script_guard.ts's location.origin is not reachable from the creative bundle.
• enableDebugFromEnv's localStorage access is try-wrapped, so the opaque origin's SecurityError cannot break guard installation.
• The /auction path yields an empty adm on rejection and carries no cache coordinates, so it has no parallel to the build_bid_map bypass.
• The 1 MiB cap is applied after ${AUCTION_PRICE} expansion, so macro-driven growth cannot slip past it.

CI Status

• fmt: PASS
• clippy / cargo check (cloudflare native + wasm32-unknown-unknown, spin native + wasm32-wasip1): PASS
• rust tests (fastly, axum, cloudflare, spin, ts CLI, cross-adapter parity): PASS
• js tests (vitest) + format-typescript + format-docs: PASS
• integration tests (browser, Fastly EC lifecycle, prepare artifacts): PASS

[prk-Jr]

❓ question — The cache fallback is now suppressed for accepted creatives too, not just rejected ones

The previous round asked for the PBS Cache fallback to be suppressed when processing rejects a supplied creative. This block also removes hb_cache_host/hb_cache_path whenever a creative is supplied and accepted — the normal PBS path — since only the None arm re-inserts them.

Those two keys are not read solely by the bridge. They are set as GPT slot targeting:

// crates/trusted-server-js/lib/src/integrations/gpt/index.ts:544
TS_BID_TARGETING_KEYS.forEach((key) => {
  if (bid[key]) gptSlot.setTargeting(key, String(bid[key]!));
});

which is what the Prebid Universal Creative uses to fetch the cached creative on its own. Where the bridge declines to serve — notably the matchedBid.hb_adid !== adId early return at gpt/index.ts:1049, a case this codebase has already hit with non-unique hb_adid — the slot previously still rendered from PBS Cache and now has no render source at all.

The security argument covers this case as well: the cached copy is the raw adm, so serving it would bypass processing whether or not the inline creative was accepted. So this may well be deliberate. If it is, the CHANGELOG entry and PR description should say so — both currently describe only the rejected case:

a supplied creative that processing rejects no longer falls back to PBS Cache coordinates

Could you confirm the accepted-case suppression is intended, and if so extend that sentence to cover it?

[prk-Jr]

🤔 thinking — The about:srcdoc branch is never executed by any test

The srcdoc fix rests on document.baseURI resolving to the parent document's URL inside a sandboxed srcdoc iframe. Per spec that inheritance holds (an about:srcdoc document's fallback base URL comes from its parent), so I read this as a coverage gap rather than a defect.

But the jsdom tests added in this commit only override window.origin; jsdom's document.baseURI is an ordinary http:// URL, so base !== 'about:srcdoc' is always true and the location.href fallback path never runs. The failure reproduced in headless Chromium last round is therefore still not covered by anything.

The deferral note on the click.ts thread says a real srcdoc regression test is bundled into #982 because both "need the same browser-test harness for creative iframes". A Playwright/chromium harness already runs in CI today:

.github/workflows/integration-tests.yml:156   name: browser integration tests
crates/trusted-server-integration-tests/browser/

So the harness cost may be lower than the deferral assumes — a single test that renders a srcdoc creative with CREATIVE_SANDBOX_TOKENS and asserts a mutated click rebuilds would pin both this and the GET 302 recovery.

[prk-Jr]

🤔 thinking — Worth naming renderGuard here so operators can tell whether #982 affects them

Dynamic resource signing is only installed when the render guard is enabled, and it is off by default:

// crates/trusted-server-js/lib/src/integrations/creative/index.ts:16
const DEFAULT_CONFIG: Required<TsCreativeConfig> = {
  clickGuard: true,
  renderGuard: false,
};

As written, the note reads as though every deployment loses dynamic resource proxying. One clause — that this applies only where renderGuard is enabled — would let an operator skip the rest of the paragraph.

[prk-Jr]

⛏ nitpick — The comment promises more than the assertion checks

the route must be registered for GET (the handler 302s)

The test only asserts status != 404, so a 400 or 500 passes — which is the exact weakness called out on the axum registration last round. The real 302 assertion lives in proxy_rebuild_get_with_origin_form_uri_redirects, and routing is genuinely all this test can check cheaply. Either drop the parenthetical or assert 302 with a validly signed tsclick as the core test does.