Repository navigation
Mask secret-named resolved parameters in results.json - #2
Merged
Merged
Conversation
results.json recorded each step's and cleanup entry's resolvedParams verbatim, so a value under a key like `password` or `userPassword` written in scenario YAML (or bound at run time) reached the file even though report.html and the workbench's run events showed `***`. That contradicts the project rule that no secret value appears in any report. JsonResultWriter now writes every result through Secrets.paramsMasked, which replaces values under secret-looking keys (Secrets.isSecret), at any depth, in steps, cleanup and skipped cleanup entries -- the rule the HTML report already applies. The masking that lived in RunSessions moved into Secrets so the events and the file share one implementation. Docs: capability matrix row "Structured results", writing-scenarios §8/§9, scenario-authoring guide, architecture README and the introduction deck no longer describe the JSON as unredacted. Tests: JsonResultWriterTest (nested, cleanup, skipped cleanup; inputs unchanged), SecretMaskTest (a by-name value the mask does not know), RunCommandTest (ldap.create with userPassword kept out of results.json and report.html end to end). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013yRxsoDBQHCgAkT7DK8KgA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
results.jsonrecorded each step's and cleanup entry'sresolvedParamsverbatim. A value under a secret-looking key — e.g. afile.writerow{ id: E1, password: hunter2-plain }or anldap.createwithuserPassword: hunter2-plain— landed inreports/<run-id>/results.jsonin plain text, whilereport.htmland the workbench's run events already showed***. The capability matrix documented this as deliberate ("JSON includes resolved parameters without redaction"), but it contradicts the project rule: no secret value in any response, event, log, report or DOM.Decision
Don't keep it. Mask by key in the JSON too, with the same rule the report uses (
Secrets.isSecret).Changes
Secrets.paramsMasked(ScenarioResult | StepResult | CleanupResult)(engine): replaces values under secret-looking keys, at any depth, in steps, cleanup and skipped cleanup entries; everything else is unchanged. This logic moved out of the workbench'sRunSessions, which now delegates, so the events and the file share one implementation.JsonResultWriter.writemasks every result as it writes the file. This one point coversRunService(idmtest runand workbench runs) and the Maven runners, since all write throughRunRecorder. In-memory results are unchanged.secrets.envvalue masking (****,SecretMask) still applies on top.writing-scenarios.md§8 (set:and descriptions) and §9 (Results);guides/scenario-authoring.md;architecture/README.md; the introduction deck. They still state that masking goes by key name only, so secrets in descriptions, messages or values under ordinary keys are written as is.Tests
JsonResultWriterTest: nested rows, headers, LDAP attributes, cleanup and skipped cleanup are masked; the input results are not changed.SecretMaskTest: a value the run's mask doesn't know, under apasswordkey, is written as***.RunCommandTest: an end-to-end run withldap.createanduserPassword: hunter2-plainkeeps the password out ofresults.jsonandreport.html.Run locally: engine (649) and wiring (78) suites pass. Runner suite: all touched tests pass. Four unrelated tests (
FolderBrowserTest,RecentWorkspacesTest, two inWorkspacesControllerTest) fail only under a POSIX locale because they use non-ASCII folder names; they pass withLANG=C.UTF-8.Note
Existing
results.jsonfiles written before this change are not rewritten and still contain plain values.Generated by Claude Code