Skip to content

Move the CHNS tests off the trace-sulfur closure edge - #44

Merged
timlichtenberg merged 8 commits into
mainfrom
tl/main-test-fixes-after-hirschmann
Oct 26, 2026
Merged

timlichtenberg merged 8 commits into
mainfrom
tl/main-test-fixes-after-hirschmann

Conversation

@timlichtenberg

Copy link
Copy Markdown
Member

Description

Since the per-element closure gate of #39, test_full_chns_converges sits at the edge of what the solver can close (with the Fischer buffer as well as with the new Hirschmann default): for H 1e20, C 1e17, N 1e17 and S 1e16 kg, no restart finds a root that closes S to its own tolerance, and the per-element gate returns the warned fallback after 100 more attempts. Whether a seed converges there depends on round-off, so on the CPU of the runner. The test then ran 2 to 3 times longer, and it timed out at 480 s in the nightly of #43 (https://github.com/FormingWorlds/CALLIOPE/actions/runs/40238902062) while it passed on main (https://github.com/FormingWorlds/CALLIOPE/actions/runs/40241498764). This moves each test away from that edge.

  • test_full_chns_converges uses S/H 3e-3, a target that converges, and asserts no fallback, each element within its tolerance, gas plus melt equal to the budget, and the surface pressure.
  • A new test keeps a trace-S case far below the edge (S/H 1e-5): it asserts the warned fallback, one warning that names S alone, the attempt count between FALLBACK_ATTEMPTS and nguess, and a state inside the CHNOS mass gate. It makes no assertion on the size of the S miss, which depends on the restart sequence.
  • The trace-sulfur fallback test pins S2_bar and S_kg_total at 0 again.
  • The example on the installation page used the edge case and promised a few hundred bar; it now uses a converging case and states the output it prints (0.36 bar), and a test runs that code block. The page names CPython 3.12 and 3.13, as pyproject.toml requires.
  • The module timeout is 780 s, set from the slowest CI duration by the rule of docs/Explanations/testing.md.
  • docs/Explanations/mass_balance.md says that a trace element far below the dominant budget can end in the warned fallback, and what the warning names.

No source change.

Validation of changes

macOS (Apple silicon), Python 3.12, tip 0ffa6e880bf836acb2d09aea5d500ef0d4d7a475:

The targets come from seed scans on macOS and on CI runners of more than one CPU type; the final tests ran only on the nightly runner above.

Checklist

  • I have followed the contributing guidelines
  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • My changes generate no new warnings or errors
  • I have checked that the tests still pass on my computer
  • I have updated the docs, as appropriate
  • I have added tests for these changes, as appropriate
  • I have checked that all dependencies have been updated, as required

With no N and a 1e14 kg S budget the returned fallback holds no S: S2_bar and S_kg_total are 0 on seeds 0 to 2, on the Hirschmann and the Fischer buffer alike.
At S/H 1e-4 the full CHNS test sits at the edge of convergence: whether a root closes S depends on the BLAS kernel of the machine, and on some CI runners the test ran past its timeout. The convergence test now uses S/H 3e-3, 10 times the smallest ratio that converged on seeds 0 to 2 on every machine tried, and asserts no fallback and every element within its tolerance. A new test at S/H 1e-5 asserts the warned fallback, a warning that names S and a state inside the CHNOS mass gate. The module timeout is 4 times the slowest CI duration, 193 s, and the mass balance page names the trace-S case.
The convergence test pins P_surf against the Hirschmann value with a Fischer guard and takes its tolerances from the solver defaults. The fallback test asserts that the warning names S alone, that the solve stops after FALLBACK_ATTEMPTS and well before nguess, and pins P_surf; its docstrings state the current case, not the scan. The mass balance page places the trace-S case in the fallback step and says that the returned state keeps most of the S budget.
At S/H 1e-4 the example returned a warned fallback at 0.36 bar while the page promised a few hundred bar. It now uses S/H 3e-3, prints Surface pressure: 0.36 bar with no warning, and a test runs the code block of the page and checks the printed line and the absence of a fallback. The page names Python 3.12 or newer, as pyproject.toml requires.
The trace-S test bounds the attempts above by the fallback cap plus a few draws, the install example test asserts the Hirschmann buffer and no warning of any kind, and the convergence test keeps the P_surf pin without the range and residual checks it implies. The mass balance page says that the S kept by the fallback depends on the restart sequence, and the install page names CPython 3.12 and 3.13.
The draw of the first fallback depends on the machine, so the test bounds the attempt count by the fallback cap below and the nguess of its helper above, one named constant.
The convergence test checks the returned atmosphere and melt masses against each budget next to the solver residual. The fallback test comment says that the scalar gate bounds H, C and N and not the S kept, and the install example test asserts that no warning is captured. The mass balance page introduces the trace-S case as a fallback that returns a budget not emptied.
The docstring gives the edge between S/H 1e-4 and 1e-3 with the machines it was measured on, so the 3e-3 target sits 3 to 10 times above it, and the module comment names the nightly job and the test whose duration sets the timeout.
@timlichtenberg
timlichtenberg requested a review from a team as a code owner October 26, 2026 04:20
@timlichtenberg
timlichtenberg merged commit 97ee527 into main Oct 26, 2026
9 checks passed
@timlichtenberg
timlichtenberg deleted the tl/main-test-fixes-after-hirschmann branch October 26, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant