Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 63 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,69 @@ Always cover:
3. **Error cases** — invalid inputs, type errors.
4. **Integration** — components working together.

Naming: `test_<function_name>_<scenario>_<expected_outcome>`.
Naming: `test_<function_name>_<scenario>_<expected_outcome>`, **kept under 60
characters**. Omit any part the module or class name already carries — a test in
`test_config.py` needs no `config_` prefix — and drop `_returns_*` tails when the
scenario already implies the outcome. Keep `_raises` for error cases; the
exception type is usually redundant with it.

```
# too long — the module already says `summary_generator_stress`, and
# `_raises_runtime_error` repeats what `pytest.raises` in the body states
test_summary_generator_stress_init_strain_without_filenames_raises_runtime_error

# good
test_init_strain_without_filenames_raises
```

### Test tiers, markers, and locations

Tests are split into three tiers so the fast ones can be run without waiting on
real data or being interrupted by GUI pop-ups. **Put every new test in the tier
matching what it actually touches, and apply the marker that tier requires.**

| Tier | Location | Marker | Run with |
|---|---|---|---|
| Unit | `tests/unit/<module-path>/`, mirroring the `pyrs/` package layout | *none* | `pixi run test-unit` |
| Integration | `tests/integration/` (flat) | `@pytest.mark.integration` | `pixi run test-integration` |
| GUI | `tests/ui/` | `@pytest.mark.gui` **and** `@pytest.mark.integration` | `pixi run test-gui` |
| By-hand scripts | `tests/scripts/` | *n/a — never collected* | run the file directly |

Marker definitions, registered in `pyproject.toml`:

- **`integration`** — exercises real file I/O (`tests/data`, the `/HFIR`
archive) or a multi-component workflow.
- **`gui`** — constructs or drives Qt widgets; requires a display (xvfb or
offscreen).

Notes:

- Markers are enforced by `addopts = "--strict-markers"`: an unregistered marker
is an error, not a silent no-op. Register new markers in `pyproject.toml`.
- `test-gui` selects `-m gui`; `test-integration` selects
`-m 'integration and not gui'`. GUI tests therefore carry **both** markers —
`gui` alone would drop them from the integration tier.
- A test is a *unit* test only if it needs no real file and no widget. If it
loads a fixture file purely for convenience, prefer rewriting it against a
synthetic fixture and keeping it in the unit tier.
- Apply a whole-module marker with `pytestmark = pytest.mark.integration` rather
than decorating every function.
- `tests/util/` holds shared helper modules and fixtures, not tests of its own
(beyond tests *for* those helpers); `tests/scripts/` is excluded from
collection via `norecursedirs`.
- `test-gui` sets `QT_QPA_PLATFORM=offscreen` itself, so no window ever appears.
The full `pixi run test` does **not** — it is the task CI drives under
`xvfb-run`, and a pixi task `env` would override that. Export
`QT_QPA_PLATFORM=offscreen` yourself before running the full suite on a
workstation with a real desktop session, or it will stall on the GUI tier
until the timeout below fires — see [docs/ground_truths.md](docs/ground_truths.md).
- **Every test has a 300-second timeout** (`timeout`/`timeout_method` in
`pyproject.toml`, via `pytest-timeout`). A hang is a failure, not an infinite
wait. `timeout_method = "thread"` is deliberate: a test blocked inside Qt's C++
event loop never returns to the interpreter, so the default `signal` method
cannot interrupt it — verified. The watchdog dumps every thread's stack, which
names the blocking line. Override per-test with `@pytest.mark.timeout(N)` for a
genuinely long-running case rather than raising the global limit.

## 🔍 Code Review Process

Expand Down
57 changes: 57 additions & 0 deletions docs/ground_truths.md
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,63 @@ using that name. `grep -rn '"\.\./\|\.\(json\|csv\|h5\|xml\)"' tests/`
`open()`, `os.remove()`, etc.) is a reasonable sweep for this pattern in
other UI tests.

## `pixi run test` hangs on the GUI tier under an interactive display (2026-09)

Running the full suite on a workstation with a real desktop session
(`DISPLAY=:0` plus a live Wayland compositor) and **no** `QT_QPA_PLATFORM`
override hangs indefinitely at the first GUI test,
`tests/ui/test_calibration_ui.py`. It is not slow — it is blocked. The
process sits at ~6% CPU with `wchan = poll_schedule_timeout`, holding the
compositor's cursor-shm, `mime.cache` and `icon-theme.cache` file
descriptors open: the signature of a real, mapped Qt window spinning its
event loop waiting for an interaction that never arrives. pytest's stdout
is block-buffered when redirected, so the log shows nothing after the
`tests/ui/test_calibration_ui.py` line and the run looks merely slow.

Set `QT_QPA_PLATFORM=offscreen` and the same suite completes in under six
minutes. This matches what `scripts/development/run_tests.py`'s own
docstring assumes ("the offscreen Qt platform used for local runs"), but
nothing in the repo actually *sets* it, so whether a local run works
depends on the developer's desktop environment.

Note the asymmetry with the segfault entry above: `offscreen` is what
makes a local run finish, while CI's real display server (`xvfb-run` +
`xcb`) is what makes the shutdown segfault reproducible. The two failure
modes want opposite platforms, which is why neither is reliably visible
from the other's environment.

**Resolved (2026-09-18), two ways:**

1. `test-gui` now carries `env = { QT_QPA_PLATFORM = "offscreen" }` in its
pixi task definition, so it can no longer open a window regardless of
the developer's desktop. It is *not* set on the `test` task: a pixi task
`env` overrides the ambient environment unconditionally (verified — a
caller's `QT_QPA_PLATFORM=xcb` is ignored, and neither `${VAR:-default}`
in `env` nor in `cmd` expands), so setting it there would silently
defeat CI's `xvfb-run` wrapper and switch CI off the `xcb` platform
this very section is about.
2. Every test now has a 300s cap (`pytest-timeout`, configured in
`pyproject.toml`). A hang is therefore a failure with a stack dump
rather than an unbounded wait, including on the full `pixi run test`
where offscreen is not forced.

`timeout_method = "thread"` is required, not a preference. Measured
directly against a `QEventLoop().exec()` that never returns to the
interpreter: with `--timeout-method=signal` (the plugin's Unix default)
the 5s timeout passed unnoticed and an external `timeout 40` had to kill
the process; with `--timeout-method=thread` it was caught at 5s and the
dump named the exact blocking line. SIGALRM is only delivered when the
interpreter next executes bytecode, which a blocked C++ event loop never
does.

**Practical consequence:** prefer `pixi run test-unit` /
`pixi run test-integration` for day-to-day work — they deselect the `gui`
marker entirely and never open a window. (Confirmed: the only `tests/ui/`
tests the integration tier selects are `test_model`,
`test_model_multiple_files` and `test_model_from_json`, which construct
`Model()` and touch no widget.) Before the full `pixi run test`, still
export `QT_QPA_PLATFORM=offscreen` yourself.

## Uncalibrated (`Status: -1`) calibration JSON silently applied during reduction (2026-07)

`read_calibration_json_file()` in
Expand Down
Loading
Loading