Make torch and the sklearn stack optional extras - #23
Merged
Conversation
The pure-numpy processors needed none of torch, scikit-learn, river or
pandas, but installing ezmsg-learn to use e.g. the LRR re-referencer paid
for a full PyTorch install. Move the backends behind extras:
pip install ezmsg-learn # numpy-only processors
pip install "ezmsg-learn[sklearn]" # + pandas, river, scikit-learn
pip install "ezmsg-learn[torch]" # + torch
pip install "ezmsg-learn[all]" # everything
numpy, scipy and array-api-compat become declared base dependencies; they
are imported directly today and only arrived transitively. scipy has to
be a base dependency rather than an extra because model/refit_kalman.py
imports scipy.linalg at module level and reaches it via scikit-learn
today. dim_reduce belongs to the sklearn group: adaptive_decomp.py
imports sklearn.decomposition at module level.
Modules behind an extra wrap their backend import so a missing install
names the extra to install instead of raising a bare
ModuleNotFoundError. Two modules do better than a straight guard:
- util.py needs no extra at all now. Only the regressor registries touch
river/sklearn, so they are built on demand; the enums and
ClassifierMessage are pure ezmsg. Public registry names are preserved
lazily via PEP 562 __getattr__.
- collection/sample_adapt_regressor.py imports its backend lazily, so it
needs only the extra for the model_type in use -- and none at all for
model_type="kalman". It would otherwise be the one module requiring
[all].
This is a breaking change for anyone relying on `pip install ezmsg-learn`
to supply a backend; no deprecation cycle.
tests/unit/test_optional_deps.py asserts in a subprocess that no
base-install module transitively loads a backend, and that each guarded
module names its extra. The latter is only observable without the extra
installed, so it skips in the dev environment and runs for real in a new
minimal-install CI job.
The test and docs dependency-groups pull ezmsg-learn[all] so `uv sync`
and the docs build still get every backend. The docs workflow gains
--no-sync: `uv run` was re-syncing to the default groups and silently
undoing the preceding `uv sync --only-group docs`, so the environment
that was declared and the one that got built were different.
Closes #22
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #22.
The pure-numpy processors need none of
torch,scikit-learn,riverorpandas, but installingezmsg-learnto use e.g. the LRR re-referencer paid for a full PyTorch install. The backends now live behind extras:This is a breaking change for anyone relying on
pip install ezmsg-learnto supply a backend, with no deprecation cycle — as agreed on the issue, since the main downstream consumer is updated in lockstep.Two adjustments to the shape proposed in the issue
scipyhad to join the base dependencies, not an extra.model/refit_kalman.pyimportsscipy.linalg.solve_discrete_areat module level and reaches it transitively via scikit-learn today; it would have broken on a base install.numpyandarray-api-compatare declared as the issue suggested.dim_reducebelongs to the sklearn group.dim_reduce/adaptive_decomp.pyimportssklearn.decompositionat module level, andincremental_decomp.pyimports from it. The issue's module table doesn't list either.Error messages
New
src/ezmsg/learn/_optional.pybuilds the message; each module behind an extra wraps its backend import:Two modules do better than a straight guard
util.pyneeds no extra at all now. Only theADAPTIVE_REGRESSORS/STATIC_REGRESSORSregistries touch river/sklearn, so they're built on demand; the enums andClassifierMessageare pure ezmsg. The public registry names are preserved lazily via PEP 562__getattr__.collection/sample_adapt_regressor.pyimports its backend lazily, so it needs only the extra for themodel_typein use — and none at all formodel_type="kalman". It would otherwise have been the one module requiring[all].Tests and CI
tests/unit/test_optional_deps.pyasserts two properties:import torchtossr.py.minimal-installCI job that installs with no extras.The
testanddocsdependency-groups pullezmsg-learn[all], souv syncand the docs build still get every backend. (tool.uv.default-extrasisn't supported by the current uv, so the self-referential extra is the working route.)The docs workflow also gains
--no-sync:uv runwas re-syncing to the default groups and silently undoing the precedinguv sync --only-group docs, so the declared docs environment and the built one were different. The docs group now declaresezmsg-learn[all]outright instead of inheriting backends from the test group.Verification
ruff checkandruff formatclean.pip install .installs no torch/sklearn/river/pandas. All 33 optional-dep tests pass there;from ezmsg.learn.process.ssr import LRRSettings, LRRUnitworks withtorchabsent fromsys.modules; the kalman collection builds with zero backends loaded..venv: build succeeds with the same 4 pre-existing warnings, all 20 module source pages generated, and API pages carry real content across both extras (process.torch9 classes,process.rnn4,dim_reduce.adaptive_decomp10,process.ssr7).🤖 Generated with Claude Code