Repository navigation
Rewrite the Validation pages for students and correct their numbers - #45
Merged
Merged
Conversation
…s for new readers Each case now says why the check exists, what it tests, how we know the right answer and why it matters, with the reference, tolerance, test id and command in a collapsed details box and the worked numbers in a second box. The pages state what each test does not check: the janaf_H2 case checks how the code combines the coefficients with the buffer, not the coefficients against the JANAF tables, and the solve evaluates every IW buffer at 1 bar, so the Hirschmann pressure term is used only by direct calls. The validation index links the Hirschmann case under its new heading.
The solve page explains the 2 ways to call the solve and gives the round trip and the noble-gas split as cases with their inputs, results and test details; the comparison with atmodeller is named as a known limit. The structure page says that the Earth case holds by construction and checks the form and scale of the formula; the core mass from an r^2 volume is 5.6e17 kg, and the cross-reference to the PROTEUS satellite module is removed because that module does not use the Earth core fraction.
…their terms The chemistry page has a case for the NASA Glenn check of all 6 constants, and the solve page a case for the 72-case gas-only grid against atmodeller; both say that their expected offsets are a measured snapshot. The solubility page names the helium test and the abstract value 56e-5 that it pins, and says that the sulfur value is a regression check. The pages define oxygen fugacity, the IW buffer, dex and the 2 solve modes, the Hirschmann branch test is a companion line with its real bound of 0.056 dex, the structure closure test is described as a positivity check, and the index says that every page uses the case layout.
The free energy of Fe + 0.5 O2 = FeO(liq) used by OxygenFugacity.oneill is in Table 6 on p. 163 of O'Neill and Eggins (2002), Chem. Geol. 186, 151. The code comment, the test docstring and the Validation footnotes now cite that table; the paper prints no gas constant, so the comment no longer says that 8.31441 is quoted from it.
…r bounds The NASA Glenn and gas-only grid cases now say what a stored offset is, that it is recorded from a run and says nothing about which data set is right, and how large the largest one is (CH4, 0.100 and 0.109 dex). The grid case gives its reference inventory, radius and the element-budget bounds of the buffered and O-budget cases separately. The chemistry heading names 6 constants, the case links the Fischer buffer, and the solve intro names atmodeller as a reference. The oneill comment, the buffers.py docstring and the Validation page agree that the paper prints no gas constant.
…id wording For SO2, NH3 and H2S the test compares log10 Q with half of log10 K, because the code writes those constants for 2 moles of product; the chemistry case now says so and defines nu as the exponent of the oxygen fugacity. The pages no longer give a cause for the offsets from NASA Glenn and atmodeller, the grid tolerances are one check per line, and the round-trip spread is the measured 1.4e-4 over 5 starts.
The definitions of chemistry.md, oxygen_fugacity.md and solubility.md began with a space, so the docs build reported their references as unresolved although they rendered. The build now reports 148 issues, 9 fewer than main.
Each case whose reference is a regression value, a figure read or an optional code now has a Known limit line before its details box, and the pages define dex, regression check, cross-code check and atmodeller before first use. The NASA Glenn data cite McBride, Zehe and Gordon (2002) and the Fischer case cites Chase (1998). The H2S case names the coefficients it holds as a wrong fit, the Why lines name the failures each test catches, the pressure slope of the IW buffer is derived from the volume change of the solids, the grid bounds include the 10 kg floor, and the dated Last compared notes are gone. The structure test drops the satellite cross-check and gives the r^2 core as 5.6e17 kg; the oneill comment and the buffers.py docstring keep 8.31441 without a claim about the paper.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #45 +/- ##
==========================================
+ Coverage 98.90% 99.07% +0.17%
==========================================
Files 7 8 +1
Lines 913 1297 +384
Branches 167 262 +95
==========================================
+ Hits 903 1285 +382
- Misses 4 5 +1
- Partials 6 7 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
The 17 cases of the chemistry, oxygen fugacity, solubility, solve and structure pages now draw the values their tests assert, with context curves from the same code functions, and write light and dark SVGs under docs/assets/validation. Each page shows the pair after "Why it matters" with a "How to read the plot" note that names the asserted marks, and the index marks these cases as having a plot pair. The cross-code plots draw only where atmodeller is installed, like their tests; the grid plot rasterizes its 900 points to stay below the size cap.
The H2O plot draws the Dixon law from the code and marks the 2 wrong values at 100 bar that the test holds. The 1473 K plot uses a separate symbol for the Fischer and the O'Neill and Eggins values. The S2 plot is drawn after the scale guards. The rows plots call the dashed curve the wrong fit. Each plot note gives the asserted tolerance where it is too small to see, the true distance of the H2 cross on the log axis, the full miss of the wrong fits, and the meaning of each mark shape in the grid plot.
The H2 note gives both Geq values. The 1473 K note measures each distance from the circle, says that the triangle overlaps the circle, and names the 1 bar equality. The NASA Glenn note states the pass in numbers, since the error bars are about the size of a mark. The NH3 note says the plot does not draw the wrong-fit miss. Other notes give the Fischer offset tolerance, the H2S crossing interval, the gases with the 0.001 dex bound, and the unit of the structure gap, and long sentences are split. The H2S and NH3 cases call the second set of coefficients the wrong fit, as the plots do.
On a fresh runner the first plotting test builds the font cache inside its 30 s limit, and on macOS this pushed test_modified_keq_janaf_H2_matches_closed_form_at_2000K_with_oneill past the limit. The cache is now built in pytest_sessionstart, before any test starts its timer. With an empty MPLCONFIGDIR the test takes 0.6 s instead of 13 to 15 s.
Each value that a validation test asserts and its plot draws is now named once and used by both, so a changed assert value changes the plot and fails the stale check. This covers the Fischer and Hirschmann buffer values, the slope tolerance, the water and sulfur pins and wrong values, the round-trip band and O range, the core radius fraction, and the noble gas constants. The 1473 K and round-trip legends sit off the data, plus-minus and minus signs are typeset, the oxygen fugacity axes give the unit bar and the round-trip axis dex, and the noble partition label fits the figure. The notes name the dotted lines, the marks that hide each other and the meaning of the fit-plot axis, and the grid plot is described as the offset checks only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The Validation pages for chemistry, oxygen fugacity, solubility, the solve, and the structure follow one layout now, written for a student who is new to CALLIOPE: a short "What this page checks" section and, where the page uses them, a list of its terms, then per case why the check exists, what it tests, how we know the right answer and why it matters, a "Known limit" line where the case has one, a light and dark plot pair with a "How to read the plot" note, and a box with the reference, tolerance and test id. Derivations move into "Worked numbers" boxes.
test_calliope_vs_atmodeller_72_case_gas_only_gridexists.OxygenFugacity.oneill, a test docstring andscripts/cross_backend/buffers.py; the NASA Glenn data cite McBride, Zehe and Gordon (2002), NASA/TP-2002-211556. The code files change only in comments and docstrings.validation_plotfixture of proteus-mpl, from the values the test asserts; context curves come from the same code function, and the note says which marks are asserted. The committed SVGs (docs/assets/validation/, each under 60 kB) are checked against the asserted values on every test run, so a plot cannot drift from its test.tests/conftest.pybuilds the matplotlib font cache at session start, so the first plotting test does not spend its time limit on it.No change to any result. The tests change only by drawing their plots; no assert checks anything else. 14 asserts in
test_oxygen_fugacity.py,test_solubility.pyandtest_solve.pynow take the values they compare from named variables that hold the same expressions, so that the plot draws the same values.Validation of changes
macOS (Apple silicon), Python 3.12, tip
34faec00fcbdb6db99767729bb6e4a32d92481a2.src/calliope/oxygen_fugacity.py,tests/test_oxygen_fugacity.py,tests/test_structure.pyandscripts/cross_backend/buffers.pyagainsteda20cd0finds no difference; the other test files change only by their plot code.srcfirst on the path, in one process:pytest -m "(unit or smoke) and not skip" tests: 653 passed;pytest -m reference_pinned -W error::UserWarning tests: 42 passed, which includes the stale check of all plots.test_oxygen_fugacity_hirschmann_value_at_1473K_matches_published_figureset from -11.94 to -11.96, the test fails with the stale-plot message, and the same file with only a comment line added passes. Each plot takes its values from the variables the test asserts on.zensical build -c(0.0.39): 148 warnings against 157 on main; the 9 that go are onValidation/chemistry.md(1),oxygen_fugacity.md(1) andsolubility.md(7), and no page gains one.wrong_r2expression oftest_calculate_mantle_mass_recovers_wang_2018_earth_core_fractionwithcalliope.constants, is 5.570e17 kg;10**(-13152.48/2000 + 3.038586)= 2.8997e-4;OxygenFugacity('hirschmann').hirschmann(T, P_bar)at a relative 1e-6 below and above the Eq. 18 boundary gives jumps of 0.0178, 0.0141, 0.0173, 0.0089 and 0.0185 dex at 1000 to 3000 K, which the page quotes.Checklist