Integration testing for Identity Management systems — write scenarios as YAML or as Java
Tests provisioning workflows across LDAP, databases, REST APIs and SOAP services. Built on Spring Boot, JUnit and an embedded directory — the suite runs with no Docker, no external services and no manual setup.
The framework has two authoring tiers. Neither replaces the other.
Declarative (YAML). A scenario is a data file. No Java, no compilation — add a file and it runs. Intended for IdM consultants who do not write Java.
name: User provisioning to eDirectory
fixtures:
user:
generate: random-user
values: { department: Engineering }
steps:
- ldap.create:
dn: "cn=${user.userId},ou=users"
attributes:
cn: ${user.userId}
sn: ${user.lastName}
givenName: ${user.firstName}
- ldap.verify:
dn: "cn=${user.userId},ou=users"
expect:
givenName: ${user.firstName}
wait: { timeout: 30s, poll: 2s }Programmatic (Java). A JUnit test that calls the connectors directly. Use it when a scenario needs real logic — branching, computation, or an operation the step vocabulary does not cover.
OperationResult<LdapUser> result = edirectoryConnector.create(user);
assertThat(result.isSuccess()).isTrue();
boolean provisioned = testExecutor.waitUntil(
() -> databaseConnector.executeQuery("test-db", sql, userId).isSuccess(),
Duration.ofMinutes(5), Duration.ofSeconds(5));Both tiers use the same connectors underneath, so a fix in a connector reaches both.
An IdM consultant who does not write Java can run YAML scenarios from a folder of their own with
idmtest: unzip one download (it carries its own Java 25), create a workspace, point it at the lab and
run.
idmtest init my-tests
cd my-tests
idmtest validate
idmtest run --profile labSee the runner guide for installing, settings and secrets, reports and drivers.
Or do the whole job in the browser. idmtest ui starts the workbench on this computer only
(127.0.0.1, behind a session token) and opens it: create or open a workspace, edit scenarios with
autocomplete and live problems, fill in and test the lab connections, run, and read each failure as
either a finding (amber: the lab answered, not as expected) or a setup problem (red: unreachable,
credentials refused, scenario or settings to fix). It saves the same files idmtest run reads.
idmtest uiScreenshot, to be added: the main screen — the file tree with last-result badges on the left, the YAML editor with its Problems panel in the middle, and on the right the run panel with an open finding (expected vs found, the waiting history, evidence and the scenario's recent runs).
See the workbench guide.
Prerequisites: Java 25 and Maven 3.9+. That is all — Docker is not required.
./mvnw clean testThe suite starts an embedded UnboundID LDAP directory and a WireMock SOAP server on ephemeral ports, creates an H2 schema, runs 1129 tests, and tears everything down.
# One module
./mvnw -pl idm-test-engine -am test
# One test class (-am needs this flag, since sibling modules have no matching test)
./mvnw -pl idm-test-scenarios -am test -Dtest=YamlScenarioTest -Dsurefire.failIfNoSpecifiedTests=false
# One YAML scenario, or a glob-selected group (quote patterns containing * or braces)
./mvnw -pl idm-test-scenarios -am test -Dtest=YamlScenarioTest -Dsurefire.failIfNoSpecifiedTests=false \
-Dscenario='provisioning-*'
# Debug logging
./mvnw test -Dlogging.level.com.idm.test=DEBUGTwo further runners take the same YAML scenarios elsewhere; neither is part of the default suite (see Running against containers or a real lab).
Sibling modules are not installed to the local repository by default, so -pl <module> needs
-am alongside it. To skip rebuilding them while you work only on scenarios, see
running tests.
Drop a .yaml file into idm-test-scenarios/src/test/resources/scenarios/. A @TestFactory
discovers it and runs it as its own test.
| Key | Meaning |
|---|---|
name |
Required. Shown in test output and results |
description |
Optional |
fixtures |
Named test data. generate: random-user plus optional literal values that override it |
steps |
An ordered list. Each entry is a single-key map: the step type, then its parameters |
cleanup |
Optional. Steps with the same shape, run after steps whether they passed or failed; each entry is SUCCEEDED, FAILED or SKIPPED |
Actions change state; assertions check it and may retry.
| Step | Kind | Required | Also accepts |
|---|---|---|---|
ldap.create |
action | dn |
attributes (may include objectClass; a list writes several values), directory |
ldap.update |
action | dn, attributes (a list writes several values, [] removes the attribute, {add, remove} changes single values) |
directory |
ldap.delete |
action | dn |
directory |
ldap.verify |
assertion | dn, and exactly one of expect (a value, a list as the exact set in any order, or {equals, contains, excludes, count, matches, ignoreCase}) or exists |
wait, directory |
db.execute |
action | database, statement |
params |
db.verify |
assertion | database, query, expect |
params, wait |
file.write |
action | folder, file, rows |
columns, delimiter, quote, encoding, lineEnding, header |
file.verify |
assertion | folder, file (an exact name or a glob), expect (rowCount and/or row) |
where, delimiter, quote, encoding, header, wait |
file.delete |
action | folder, file |
— |
exec.run |
action | command (a registered command's name) |
params |
exec.verify |
assertion | command, expect (exitCode, stdout, stderr) |
params, wait |
roles.grant |
action | user, roles |
reason |
roles.revoke |
action | assignments |
— |
resources.grant |
action | user, resource |
reason |
resources.revoke |
action | user, resource |
reason |
rest.request |
action | target, method, path |
body, bodyFrom, headers |
rest.verify |
assertion | target, path, expect |
method, headers, wait |
http.request |
action | target, method, path |
headers, body (a raw string: JSON, XML, a SOAP envelope) |
http.verify |
assertion | target, method, path, check |
headers, body, wait (JavaScript conditions on response, including its parsed JSON and XML) |
Every step also accepts as: to capture its result and description: to say why the step exists. rest.* and http.request address a named
target (idm.test.rest.<target>.base-url) and authenticate with that target's settings — basic,
OAuth2 client credentials, or none — unless the step writes its own Authorization header.
exec.* run a command registered under idm.test.commands.<name>, and only once its definition is
approved on the machine (idmtest commands approve, or the workbench's lab panel). The
authoring guide walks through it. The
authoring reference documents every parameter, capture and failure mode.
Waiting is a property of assertions, not a step of its own: you never wait for its own sake, you
wait for something you were going to assert anyway. Any assertion accepts wait.
wait: { timeout: 30s, poll: 2s } # explicit — durations take s, m or h
wait: true # use the configured defaults${path} resolves against fixtures, set: variables and as: captures. Property paths only —
no arithmetic, conditionals or function calls.
When a value needs computing, a whole value can be a sandboxed JavaScript expression:
"js:{ user.firstName.toUpperCase() }". It reads scope names directly, cannot reach Java classes
or connectors, and a result containing undefined anywhere is an error rather than a silent
null. set: binds such values to names; as: captures a step's result. When a scenario needs
real logic — branching, loops, an operation the vocabulary lacks — it belongs in the Java tier.
random-user produces userId, firstName, lastName, email, department, jobTitle,
phoneNumber and employeeId.
Every scenario file is parsed and validated before the first step executes, because a scenario
can spend minutes waiting for provisioning and discovering a typo afterwards is expensive. A
scenario is rejected at load for an unknown step type, a missing required parameter, an unknown
parameter, a reference to a name that is never bound (in ${...} or inside js:{...}), an
unknown generate: value, a js:{...} that does not compile, or a malformed wait:. Every message names the file and the step.
That last point matters more than it sounds. Because steps declare their parameter names, a
misspelled expct: fails at load rather than producing a green test that verified nothing.
Each run writes idm-test-scenarios/target/scenario-results.json — every scenario, every step,
the parameters after interpolation, and for a step that waited, how many attempts it made and
what it last observed. ScenarioEngine itself renders nothing; the runners write both this JSON file and
the HTML report from its results.
Each run also writes a self-contained HTML report beside it (scenario-report.html, real-environment-report.html, multi-directory-report.html) — open it in a browser to read or share the run. Each step's evidence — the HTTP or SOAP exchange, the SQL and its rows, or the LDAP entry — is shown under it, with secrets masked.
For a file to attach to a ticket, idmtest report <run-folder> --pdf summary|detailed renders a run as a PDF on request (see the runner guide); the workbench offers the same under Download PDF.
The report and messages are in English by default; set IDM_TEST_LOCALE=es (or idm.test.report.locale: es) for Spanish. See Report language.
idm-test-framework/
├── idm-test-core/ TestExecutor, OperationResult, ErrorType
├── idm-test-utils/ Test data generators, custom assertions, EmbeddedLdapServer
├── idm-test-engine/ YAML scenario engine — loader, validator, executor, step SPI
├── idm-connector-ldap/ eDirectory/LDAP connector
├── idm-connector-database/ Multi-database JDBC connector (raw SQL)
├── idm-connector-rest/ REST connector, JDK HTTP client, target authentication (basic, OAuth2)
├── idm-connector-soap/ Role and Resource service connectors (NetIQ SOAP)
├── idm-connector-file/ Delimited text files in named folders (Delimited Text driver)
├── idm-connector-exec/ Registered local commands, no shell (dxcmd, scripts)
├── idm-connector-idm/ Placeholder — no sources yet
├── idm-test-scenarios/ Java scenario tests, YAML scenarios, the JUnit adapter
├── docker-compose.yml Optional local services; the test suite does not use them
└── docs/ See Documentation below
- Spring's built-in clients — RestClient, JdbcTemplate, Spring Security's OAuth2 client; generated JAX-WS ports for SOAP. No custom wrappers and no custom token handling.
- Raw SQL for database validation — no JPA or Hibernate entities.
- Integration-first — tests exercise real protocol implementations, not mocks of our own code.
- Two tiers, one set of connectors — the engine is a consumer of the connectors, exactly as a Java test is.
Three properties are enforced rather than merely intended:
It takes no Spring dependency. idm-test-engine never creates an application context or
reads configuration; it receives its dependencies by constructor. A test walks the module's
sources and fails on any org.springframework import. That is what keeps a future CLI a wrapper
rather than a rewrite.
It interpolates before calling a step, and owns the retry loop. Step implementations receive fully resolved parameters and never think about waiting, which keeps each new step type to roughly forty lines around one connector call.
It renders nothing. run() returns a ScenarioResult; the JSON writer and the JUnit adapter
are both adapters over that one shape.
run() always returns a result rather than throwing. It infers no teardown of its own: an author
who needs cleanup writes it explicitly as a top-level cleanup: block, which runs after the steps,
pass or fail, in declaration order. A FAILED cleanup entry fails the scenario; a step failure
still leads the failure summary. See
docs/specs/2026-09-24-explicit-cleanup-design.md.
| Concern | Provided by |
|---|---|
| LDAP directory | Embedded UnboundID, ephemeral port, standard schema extended with eDirectory's startDate, loginDisabled and c |
| Databases | H2 in-memory, schema created at context startup |
| SOAP and REST services | WireMock on a dynamic port, stubs under src/test/resources/wiremock/ |
Verified on 2026-10-20, after ITF-56 (the idmtest ui workbench), with ./mvnw -B verify (0 failures or errors; the Java totals below, plus 181 front-end tests run by Vitest in the runner's build):
| Module | Tests |
|---|---|
| idm-test-core | 37 |
| idm-test-utils | 51 |
| idm-test-engine | 645 |
| idm-connector-ldap | 77 |
| idm-connector-database | 25 |
| idm-connector-rest | 67 |
| idm-connector-soap | 29 |
| idm-test-wiring | 76 |
| idm-test-scenarios | 96 |
| idm-test-runner | 304 |
| Total | 1407 |
The scenarios module holds the Java workflow tests (provisioning, role assignment, modification, deprovisioning and multi-system flows), a corpus of 22 YAML scenarios (21 converted from them, and a delimited text file round trip), and the tests of the runners and their guards.
Worth being explicit about: the default suite simulates IdM. The Java and YAML scenarios write directly to the embedded LDAP, H2 and WireMock and assert the result, so they exercise the framework end to end but validate no real Identity Manager. The runners below are how the same scenarios reach real systems.
| Runner | Runs | Needs |
|---|---|---|
YamlScenarioTest |
the corpus in scenarios/, embedded |
nothing — part of the default suite |
ContainerScenarios |
the same corpus against OpenLDAP, PostgreSQL and WireMock | Docker; optional, not in the default suite |
RealEnvironmentScenarios |
your scenarios in scenarios-real/ against a real lab |
a profile and credentials for your environment |
./mvnw test -pl idm-test-scenarios -am -Dtest=ContainerScenarios -Dsurefire.failIfNoSpecifiedTests=false
./mvnw test -pl idm-test-scenarios -am -Dtest=RealEnvironmentScenarios -Dsurefire.failIfNoSpecifiedTests=falseBoth accept -Dscenario=<glob>. The real-environment runner checks only what the selected
scenarios use, and refuses to start, before any scenario runs, when it would test nothing real:
the embedded profile, an empty directory, or — for repositories the scenarios use — an unset or
placeholder LDAP/SOAP URL, REST target, database, or credentials. See
scenarios-real/README.md.
| Ticket | Work |
|---|---|
| ITF-16 | Oracle-compatible connection test query for database pools |
| ITF-17 | Stop the runners starting embedded LDAP and WireMock as a side effect |
| ITF-18 | Check ContainerScenarios' "Docker is not available" message without Docker |
docker-compose.yml starts OpenLDAP, phpLDAPadmin, WireMock and PostgreSQL for manual
exploration. The test suite does not use them — it runs against embedded services. These are
here for poking at a directory by hand, and for the eventual move to real infrastructure.
docker-compose up -d # phpLDAPadmin at http://localhost:8080
docker-compose downOrganised by how much you should trust it.
Project introduction and architecture:
- Documentation index — reading paths for coworkers, test authors, and maintainers
- Project overview — purpose, users, and a simple scenario
- Architecture guide — system context, modules, and execution diagrams
- Capability baseline — implemented capabilities, planned work, and verification evidence
- Scenario authoring — YAML and Java examples
- Running scenarios with
idmtest— install, workspace, settings, reports - The workbench —
idmtest ui: editing, the lab panel, runs, findings - Architecture decisions — design choices and their rationale
- Team presentation — PowerPoint, PDF, and speaker notes
Reference — maintained:
- Writing scenarios — every step, parameter and rule of the YAML tier
- Specification — architecture and design decisions
- Configuration — connectors, databases, endpoints
- Running tests — test execution guide
- CLAUDE.md — development guidelines
Specs and plans — design documents for in-flight work:
- Declarative scenario engine
- Green test baseline
- docs/specs/ — one design per ticket since ITF-12, e.g. explicit cleanup, http.request, HTTP authentication
- docs/plans/ — task breakdowns
History — frozen, partly inaccurate, kept for the record:
- docs/history/ — the banner there says which reports are wrong
| Concern | Choice | Version |
|---|---|---|
| Language | Java | 25 |
| Framework | Spring Boot | 4.1.1 |
| Testing | JUnit 6, AssertJ | 6.0.3, 3.27.7 |
| LDAP | Spring LDAP, UnboundID SDK | 4.1.1, 7.0.1 |
| Database | Spring JDBC, H2 | 2.4.240 |
| SOAP | JAX-WS, JAXB | — |
| YAML/JSON | Jackson | 3.1.5 |
| Expressions | Mozilla Rhino | 1.8.0 |
| HTTP | JDK HttpClient via Spring RestClient; Spring Security OAuth2 client | — |
| Containers (optional runner) | Testcontainers | 2.0.5 |
| Mocking | WireMock | 3.10.0 |
No Lombok — Java records instead.