Skip to content

fix(redaction): mask a bare Bearer token, short named values and the token name - #39264

Draft
chenmingwei23 wants to merge 2 commits into
mainfrom
mkt-app/redactor-gaps
Draft

chenmingwei23 wants to merge 2 commits into
mainfrom
mkt-app/redactor-gaps

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The redactor masks a bare Bearer <token>, shorter values after a listed credential name, and the bare name token, which it used to leave visible.

Today a Bearer <token> with no Authorization name before it, a 6 or 7 character value after a listed name, and a value after the bare name token all pass through unmasked. A long token cut by a trailing ellipsis would also pass if the ellipsis exemption were applied to every length.

Why it matters

Frozen. Credentials in logs and tool output that look like these shapes stay readable to anyone who reads the output.

Not a goal

  • Values of 1 to 5 characters after a listed name stay visible.
  • Short curl -u passwords (1 to 3 characters) stay visible.
  • No new credential formats beyond the shapes above.

What changed (motivation → approach → change)

Each shape was a gap in the structured redactor's header and known-key readers.

  • A new Bearer <token> reader masks the token; words after Bearer ("Bearer of bad news") stay.
  • The literal floor after a listed name goes from 8 to 6 characters.
  • token joins the listed credential names, with the same checks as password (placeholders, code and plain words stay; tokenizer=fast and token=words like this stay).
  • A value ending in ... or an ellipsis character stays visible only when what precedes the ellipsis is shorter than 8 characters. A long token cut by an ellipsis is still masked.

Merge order: after #38807.

Backwards compatibility

Compatible: the change only masks more values; nothing that is accepted today is rejected, renamed or removed.

Removes nothing: no field, key or function is removed; more text is masked.

Tests

Two commits: the tests (new tests that fail on main, plus updates to existing tests that pinned a bare Bearer value, a bare token name or a token= spelling as visible; a Basic credential stands in where a test needs a value masked only beside its key; TOKEN moves to the "names read" list), then the change.

  • Bare Bearer token is masked; "Bearer of bad news" and "the Bearer authentication scheme" stay.
  • A 7 character value after password, secret and token is masked; password=x1 stays.
  • sk-... stays for password and token; a long synthetic token cut by ... or the ellipsis character is masked.
  • tokenizer=fast and token=words like this stay.

Proof: not provable -- other: prove.py cannot run on the worker machine (no test runner installed); CI runs the tests.

Manual verification

N/A -- unit coverage sufficient; the readers are pure functions of the input text.

Real-machine verification

Not run on a separate machine; the change is text-only redaction logic covered by the tests above.

Related Issues

None.

Sibling sweep

Swept: git grep -nE "LITERAL_MIN|_NAMELESS_READERS" -- src/
Siblings: none -- the floor and the reader set are each defined once in the structured redactor.

Pattern harvest

Not generalizable: each shape was a missing case in a table-driven reader, not a repeated code pattern.

Checklist

  • A Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • src/kiro_crew/security/redaction_structured/headers.py

  • src/kiro_crew/security/redaction_structured/known_key.py

  • src/kiro_crew/security/redaction_structured/table.py

  • @chenmingwei23 comment: does not name the current head 3c17321; post it again for this head

How to clear this check:

  1. The PR author posts the template below as a PR comment, with
    every field filled (N/A needs a reason). Keep the
    Head commit line: it must name the current head SHA.
  2. The check turns green within a minute, with no push.
  3. A new push resets this check: post it again for the new head.

Only the author's comment counts this way. A reviewer can instead
paste the filled template into an "Approve" review on this head.
Merging still needs a reviewer's approval.

## Sensitive change review
- Head commit: 3c173218c016c87b61a9d35f0630344d261922ac
- What rule changed:
- Before -> after (what was allowed/blocked, now):
- Worst case if wrong (breaks commands / leaks secret / locks user out):
- Evidence checked (which sandbox result proves it):
- No regression:
  - Tools still work (which tools/commands were run, result):
  - Backward compatible (old config / old data / old callers still work, how checked):
  - Existing tests (which suites passed on this head):
- How to undo:

@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Nov 18, 2026
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 3c173218c016c87b61a9d35f0630344d261922ac touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 3c173218c016c87b61a9d35f0630344d261922ac — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: these are new rows in the existing structured redactor, they only mask more, and a revert undoes them.

Notes

  • The description never mentions the mcp_discovery_server_info.py change. redact_mcp_error now runs the configured-value scrubber before redact_credentials, so that the new bare-Bearer row cannot mask part of a configured header value before the exact match looks for it. Add a line about it under "What changed".
  • test_a_named_token_after_a_plain_sentence_end_does_not_stand_alone now pins a known gap: token=<value> written straight after a . stays visible. Name that gap in security.md next to the other known gaps.

[DESIGN-REVIEWED] 3c17321

Description read

[DESCRIPTION-READ] 7712b499182f5c91c3d000f86e7aaaeffa136283b1e5302ad7f6c5f520874656

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 3c173218c016c87b61a9d35f0630344d261922ac — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Every item closes a credential shape that used to leak, or follows directly from a fix that does. The one undeclared rider, the MCP reorder, is needed by the bare Bearer reader.

Notes

  • Item 5 — undeclared: the description never says that redact_mcp_error now runs the exact configured-value scrubber before the generic scanners ("They run in this order, so a generic match never masks only part of a configured value"). Without the reorder, the new bare Bearer reader would mask a configured header value first, and the exact scrubber would no longer find it.

What this change ships

Inventory (6 items) — 5 justified

Intent: keep bearer tokens, short labelled secrets and token= values from showing up readable in logs and tool output. This is a FIX.

  1. A Bearer <token> with no Authorization name before it is now masked; prose such as "Bearer of bad news" stays — justified
  2. Values of 6 or 7 characters after a listed credential name are now masked (the floor drops from 8 to 6) — justified
  3. The bare name token now masks its value, which also covers entity-spelled URL token= forms — justified
  4. An elided value like sk-... stays visible only when the part before the ellipsis is under 8 characters; a long token cut by an ellipsis is masked — justified
  5. MCP error redaction now masks the configured header values before the generic scanners run — undeclared
  6. Test fixtures and docstrings now use a Basic credential, or a shorter hunter2, where they had relied on the old gaps — justified

[FIRST-PRINCIPLES-REVIEWED] 3c17321

Description read

[DESCRIPTION-READ] 275a310f33371c1ac1fa9a94745e9f628e3b4a852fd59c00ff3e5e638b4e0b45

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 02c95791b1c2af664dee0bf7d8f66e0844a60e57 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 02c9579

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 02c95791b1c2af664dee0bf7d8f66e0844a60e57 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 18, 2026
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — 🔴 changes requested (blocking)

Reviewed 3c173218c016c87b61a9d35f0630344d261922ac — this comment is updated in place on each push.

Blocking: this change moves the exact-match scrub of configured header values ahead of the generic credential scanners. A short, harmless header value such as "2" now splits a credential reflected in a probe error into pieces, so the generic scanners no longer catch it and the credential reaches serialized output.

BLOCKING — src/kiro_crew/mcp_discovery_server_info.py:248

    if values:
        error = _scrub_configured_values(error, values)
    error, _ = redact_exfiltration_urls(error)
    error, _ = redact_credentials(error)

Trigger: an MCP server is configured with {"Authorization": "Bearer …", "X-Api-Version": "2"}, and its probe error reflects a presigned URL …?X-Amz-Signature=9f2c1e2a7b…&X-Amz-Expires=60. The path is redact_mcp_error → _scrub_configured_values. Every full header value is a bare, case-insensitive substring match with no length floor (values[value] = False), so each 2 in the signature becomes [REDACTED: credential]. That label contains a space, so redact_exfiltration_urls / redact_credentials then mask only the first piece of the signature.
Outcome: I reproduced this. The output was X-Amz-Signature=[REDACTED: credential]: credential]c1e[REDACTED: credential]a7b[REDACTED: credential]d4e5f6a7b8c9d0e1f[REDACTED: credential]…. That is the whole signature except its 2s, which are the known header value, and it reaches to_dict, the probe cache and the probe endpoints. Without the short header, the same input redacts to X-Amz-Signature=[REDACTED: credential]&X-Amz-Expires=60. That matches what the old order (scanners first) produced in every case.
Anchor: residual/security
Fix: run the generic scanners on the raw text as well as after the configured scrub (scanners → configured scrub → scanners). Or apply the unboundaried bare match only to values at least _MCP_CREDENTIAL_UNANCHORED_MIN_LENGTH long, and require boundaries for shorter full header values.

[BLOCK-MERGE] 3c17321
[OPUS-REVIEWED] 3c17321

Verdict parsed from the review's SHA-scoped output markers for commit 3c173218c016c87b61a9d35f0630344d261922ac.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 3c173218c016c87b61a9d35f0630344d261922ac false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- ✅ PASS

Which legitimate operations does 3c173218c016c87b61a9d35f0630344d261922ac newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: adjudicated, zero confirmed regressions.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 12 classified, 0 newly refused, 12 still allowed.
  • ubuntu-latest (platform posix): 12 classified, 0 newly refused, 12 still allowed.
  • windows-latest (platform windows): 7 classified, 0 newly refused, 7 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: PASS

Reason: The tightening is in the output credential redactor (redaction_structured). None of the three shell deny tiers calls that rule table: deny_gate uses it only to scrub log clips, and the exfil tier matches the separate _CREDENTIAL_PATTERNS regex. So the shell candidates should come back unchanged. Several refusal and withholding paths downstream of redact_credentials do get stricter, and none of them can be classified here; they are named under UNADJUDICATED.

TIGHTENING: The credential rule table masks more values. It adds a bare-Bearer reader, + "id": "http.bearer_bare", ... "reader": "bearer", "min_length": 8. It adds + "token", to the labelled.secret names. It lowers the literal floor with -LITERAL_MIN = 8 / +LITERAL_MIN = 6. It also reorders redact_mcp_error so that + error = _scrub_configured_values(error, values) runs before the generic scanners. The one threat: a credential value showing in output without its Authorization name before it (for example Bearer <token> in a logged request line), or a short token= / password= value staying visible.

ADJUDICATED: 17 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE: The diff assumes nothing about paths, separators or interpreters in a shell command line. Its only boundaries are text shapes: Bearer followed by blanks, token=, short literals, and %-escapes inside a token. I probed those shapes in each platform's way of spelling an environment-supplied credential:

  • posix: $VAR inside double quotes, through curl -H, a URL ?token= and docker login --password-stdin.
  • windows: %VAR% (cmd.exe) and $env:VAR (PowerShell), with curl.exe, Select-String and .venv\Scripts\python.exe spellings.
  • any: gh auth status and git log --grep=bearer.

The new reader stops a value at _value/_keep. Whether that stopping point treats %VAR% and $env:VAR the same as $VAR matters only for the redaction sinks below, which no leg can classify.

UNADJUDICATED:

  • Blocked-link card withholding. Guard: security/blocked_links.py _credential_clean. Call path: a flagged URL → _withheld → redact_credentials(value) on the URL and each decoded form; any change means no "Open once"/"Copy". Newly withheld inputs I would have submitted:
    • an ordinary URL such as https://h.example/p?a=1&amp;token=<16-char opaque id>, where token= is now a known-key name;
    • a URL whose query holds password= followed by a 6–7 character value;
    • a docs link whose text holds Bearer followed by a non-word identifier.
  • Command-source card pre-fill. Guard: security/credential_sources.py _is_clean. Call path: the source a tool call ran → _is_clean → refuses(value, redact_credentials); any change means the card neither shows nor pre-fills the command. Inputs: a shell command carrying Bearer and an env-var reference ($API_TOKEN, %API_TOKEN%, $env:API_TOKEN), and token= followed by a 6–7 character non-word value. Whether _bearer_bare treats a $/% reference as a token decides this, and only the Python redactor can answer it, not the shell classifier.
  • Skill packaging content scan. Guard: the producer's content scan, reached from aws_control/crew/packaging resolve_crew. The mutation test fixture had to change SECRET=hunter2 to SECRET=hunt2 because the 7-character value now redacts. Candidate inputs: a skill file holding password= or token= followed by a 6–7 character example value, or a Bearer line followed by a lowercase identifier in an example HTTP transcript. Such a skill now gets the content refusal where before only the filename rule applied.
  • MCP error and probe display. Guard: mcp_discovery_server_info.redact_mcp_error, now run with the configured values first. This masks output and does not refuse, so nothing is lost; noted only because the order changed.
  • Narrowing (argued, not confirmed). _capitalised_word keeps a single capitalised word, but Bearer followed by a lowercase non-dictionary identifier of 8+ characters is masked. That covers text like Bearer realm_name in a WWW-Authenticate echo, and those same masks feed the two refusal guards above.

[SCOPE-REVIEWED] 3c17321

@chenmingwei23
chenmingwei23 force-pushed the mkt-app/redactor-gaps branch 2 times, most recently from bfb27cf to de50ae8 Compare November 19, 2026 00:27
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 19, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: de50ae8
  • What rule changed: The structured redactor masks more written credentials. (1) A new reader masks the token after a bare Bearer word (no Authorization name before it); a word made only of letters that is capitalised or all upper case after Bearer is left as written. (2) The length floor for a written value after a listed credential name goes from 8 to 6 characters. (3) The bare name token joins the listed credential names, with the same value checks as password. (4) A value ending in ... or an ellipsis character stays visible only when the text before the ellipsis is shorter than 8 characters. Files: security/redaction_structured/headers.py, known_key.py, table.py.
  • Before -> after (what was allowed/blocked, now): Before: Bearer <token> alone, a 6 or 7 character value after a listed name, and token=<value> stayed visible. After: all three are masked. Words after Bearer, 1 to 5 character values, tokenizer=fast and token=words like this stay visible. An elided example such as sk-... stays visible; a long token cut by an ellipsis is masked.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): Too wide: ordinary text after token= or bearer is shown as [REDACTED] in logs and chat output (text is hidden, nothing is blocked, no command fails). Too narrow: a value the new rules should mask stays readable, which is the state before this change. The change adds no new way to leak a secret and removes no existing masking.
  • Evidence checked (which sandbox result proves it): The pull request checks on this head (https://github.com/kirodotdev/KiroCrew/pull/39264/checks). New tests in test_redaction_structured_names.py fail on main and pass on this head: bare Bearer masked, prose after Bearer kept, 7 character values after password/secret/token masked, sk-... kept, long token cut by an ellipsis masked, tokenizer=fast and token=words like this kept. A scripted check of the redactor on main source showed 5 of 12 cases wrong and 0 of 12 with the change.
  • No regression:
    • Tools still work (which tools/commands were run, result): Backend test shards on Python 3.12 and Windows passed on this head. No tool or command is blocked by this change (it only changes what text is masked).
    • Backward compatible (old config / old data / old callers still work, how checked): No config key, data format or function signature changed. Callers get more masking only. Existing tests that pinned the old output were updated in the tests commit (f103fbe), each to the new rule; this includes one fixture value in the aws_control packaging test (a 7 character value changed to 5 characters, because 7 characters are now masked). The other existing redaction tests are unchanged.
    • Existing tests (which suites passed on this head): Every Backend Tests shard (Python 3.12 shards 1-8 and Windows shards 1-8) passed on this head. Suites touched: test_redaction_structured, test_redaction_structured_names, test_security_token_params, test_log_redaction, test_display_split_safety, test_chat_runner_redact_across_tool_boundary, test_agent_panel_routes_record, test_cron_script_sessions, test_mediated_request_dispatch, test_mcp_probe_treadmill, aws_control packaging test_producer.
  • How to undo: Revert commit de50ae8 (the three source files return to main; the tests in f103fbe then need reverting with it).

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 19, 2026
…n=, update tests that pinned them as visible

Adds tests for a bare 'Bearer <token>' (words after Bearer stay), a 6 to 7 character value after a listed name, the token name with benign pins, and a long token cut by an ellipsis (still masked) versus a short elided example (kept).

Updates existing tests that pinned the old output as visible: a Basic credential stands in where a test needs a value masked only beside its key, token= spellings now pin the masked value, a 7 character fixture value becomes 5 characters, and TOKEN moves to the list of names that are read.
…token name

A 'Bearer <token>' with no Authorization name before it is masked whole, including a percent-escaped part of the token; words after Bearer stay, including a capitalised or upper case word. A written value of 6 or more characters after a listed credential name is masked (was 8). The bare name token is listed. A value ending in an ellipsis stays only when what precedes the ellipsis is shorter than 8 characters, so a long token cut by an ellipsis is still masked.

The probe-error scrubber now masks the configured header values first and then runs the generic scanners, so a generic match can no longer mask only part of a configured value before the exact scrubber looks for it. The security spec and the MCP architecture note describe the bare Bearer token and that order.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant