[pepaooo]

Why

Many IdM projects bring identities in through the NetIQ Identity Manager Driver for Delimited Text, for example an HR export as CSV. Until now a scenario could neither drop a file for the driver nor check a file the driver writes. This PR adds three step types so that "file in, then check LDAP, the database or AD" (and the reverse) can be tested end to end.

Design: docs/specs/2026-11-03-delimited-text-file-steps-design.md · Plan: docs/plans/2026-11-03-itf-73-delimited-text-file-steps.md

What

steps:
  - file.write:
      folder: hr-in
      file: "hr-${user.userId}.csv"
      rows:
        - { employeeId: "${user.userId}", email: "${user.email}" }
      as: written
  - file.verify:
      folder: hr-out
      file: "hr-out-*.csv"
      where: { employeeId: "${user.userId}" }
      expect: { rowCount: 1, row: { status: provisioned } }
      wait: { timeout: 2m, poll: 5s }
cleanup:
  - file.delete: { folder: hr-in, file: "${written.name}" }

Folders

• Folders are declared by name under idm.test.files.<name>, with path, an access setting (read / write / read-write, default read) and a file format.
• A scenario never gives a path. A file name that would leave its folder is rejected: when the scenario is validated if it's written literally, and when the step runs if it's built from ${...}.
• Messages, results, reports and the workbench name the folder by its key, never by its path.

file.write

• Writes under a temporary name, then renames the file in one atomic step, so a polling driver never reads half a file.
• Never overwrites an existing file. Fails if a row has a key that isn't one of the columns, so a typo is never dropped silently.

file.verify

• Reads the newest matching file modified since the scenario started, so a file from an earlier run can't make it pass. A per-folder clock-skew allows for a share's clock.
• Selects rows with where: and checks rowCount / row, the same way db.verify does.
• Retried: no file yet, a file that doesn't parse yet, rows that differ. Final: configuration problems.

file.delete

• Treats a file that is already gone as success, because the driver usually removes its input.

How

• New module idm-connector-file: DelimitedFileConnector (Apache Commons CSV). Spring appears only in its config package.
• Engine:
  • The three steps.
  • FileEvidence: masked and size-limited, and it always keeps the rows where: selected.
  • StepContext: gives a step the scenario's start time, through a default execute overload.
  • StepType.literalProblem: a validation hook for literal parameter values.
  • RowExpectations: db.verify's expectation checks, now shared. db.verify's messages are unchanged.
  • Messages in English and Spanish.
• Wiring: 17 registered step types. ScenarioNeeds gains the folders read and written. A real-environment guard refuses an undeclared or missing folder, or one with too little access as declared or as the operating system allows. The guard creates nothing in the folder, because a driver may be watching it.
• Runner and workbench:
  • Folders are lab systems (folder:<name>). Saving refuses a value the runner could not bind.
  • The connection test uses a probe file that is deleted at once.
  • The finding panel shows file evidence, without the folder's path.
  • SecretMask covers file evidence.
• Docs: authoring guide and reference, configuration (idm.test.files), workbench guide, run guards, capability matrix, README, CLAUDE.md.

Testing

• New tests: connector tests over real temporary folders (containment, symbolic links, atomic write, choosing the file to read, malformed files, access); FileStepsTest, including a wait through the engine while another thread writes the file; guard tests; lab and connection-test tests; idmtest run tests; Vitest tests.
• Default-suite scenario files-delimited-round-trip.yaml: writes, waits and checks with where:, then deletes in cleanup, against a fresh folder under target/. DelimitedFileScenarioTest then checks that the folder is left empty and that no path appears in the evidence.
• Deliberate breaks: I broke each important safeguard on purpose to confirm the tests catch it.
• Hand-run with the built jar: an idmtest run of the round trip and a masking scenario. Neither the folder path nor the secrets.env secret appears in report.html, run.log or results.json. A workbench session confirmed the lab panel and a run.
• ./mvnw clean install passed in every module, but on JDK 21 (-Djava.version=21 -Dmaven.compiler.release=21) and with LC_ALL=C.UTF-8, because the build container had no Java 25 JDK. Please run ./mvnw clean install on Java 25 before merging.
• Not run: ContainerScenarios, which needs Docker. It now declares the scratch folder too.

Out of scope

• SFTP or SMB access by the framework itself (mount the share instead), fixed-width files, and several CSV sections in one file.
• results.json keeps step parameters unmasked by key. This is existing, documented behaviour, so a literal password in a file.write row shows up there, as userPassword in ldap.create already does. I've suggested a separate task for it.

---

Generated by Claude Code