Skip to content

Support DataDeps 1.0 and GZip 0.8 - #268

Merged
CarloLucibello merged 2 commits into
JuliaML:masterfrom
farrellm:compat-datadeps-1-gzip-0.8
Oct 21, 2026
Merged

CarloLucibello merged 2 commits into
JuliaML:masterfrom
farrellm:compat-datadeps-1-gzip-0.8

Conversation

@farrellm

Copy link
Copy Markdown
Contributor

Widens two compat bounds that currently hold these packages back for downstream users: DataDeps to 1.0 and GZip to 0.8.

DataDeps 1.0 (DataDeps = "0.7, 1")

DataDeps 1.0 has two breaking changes that affect MLDatasets:

  • DataDeps.fetch_default was removed. Its replacement is DataDeps.fetch(remote_path, local_dir; ...), which also returns the local path. It is used in the graph datasets (TUDataset, TemporalBrains, Traffic, WindMillEnergy, ChickenPox, MovieLens, OGBDataset).
  • download(::DataDep, dir; ...) is now a method of Downloads.download, not Base.download. On 1.0, DataDeps.download(...) resolves to plain Base.download and throws a MethodError. This breaks datadir with a user-supplied dir and download_dep.

src/download.jl now binds datadeps_fetch / datadeps_download at load time, based on isdefined(DataDeps, :fetch_default), so both 0.7 and 1.0 work. 0.7 is kept because many other packages (Transformers, FastAI, Embeddings, PalmerPenguins, UCIData, …) still require DataDeps 0.7. Requiring 1.0 would hold MLDatasets back for their users.

GZip 0.8 (GZip = "0.5, 0.6, 0.7, 0.8")

GZip 0.8's only breaking change is that eof reports end of stream right after the last line is read. The MNIST reader only calls gzopen and reads binary data, so it isn't affected.

Testing

  • DataDeps 1.0.0 and 0.7.14, each with an empty datadeps store: TUDataset("MUTAG"), ChickenPox(), MNIST(:test; dir=...) and MLDatasets.download_dep("MNIST", dir) all download and load.
  • GZip 0.8.1 and 0.7.1: MNIST and FashionMNIST (train and test) give identical shapes, label ranges and pixel sums.
  • I haven't run the full test suite locally; it downloads every dataset.

Release request

master already has the MAT 0.12 (#256), GZip 0.7 (#257) and FixedPointNumbers 0.9 (#259) widenings, none of which have been released yet. Could this go out together with them as 0.7.22?

Dependabot hasn't proposed GZip 0.8 or DataDeps 1.0 because it is at its default limit of 5 open Julia PRs (#263–#267, all lower-bound bumps from July). Closing those would let it open new PRs again.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FEjYLhJeFY8SvquBAfpCST

farrellm and others added 2 commits October 21, 2026 00:33
DataDeps 1.0 renamed `fetch_default` to `fetch` and moved
`download(::DataDep, dir)` from `Base.download` to `Downloads.download`,
so `DataDeps.download(...)` now resolves to `Base.download` and throws a
MethodError. Pick the right functions at load time so both 0.7 and 1.0
work, and widen compat to `DataDeps = "0.7, 1"`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FEjYLhJeFY8SvquBAfpCST
The only breaking change in GZip 0.8 is `eof` after the last line, which
the MNIST reader doesn't use (it only does binary reads via `gzopen`).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FEjYLhJeFY8SvquBAfpCST
@codecov-commenter

codecov-commenter commented Oct 21, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 30.00000% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.75%. Comparing base (3d40a68) to head (f0962b2).
⚠️ Report is 21 commits behind head on master.

Files with missing lines Patch % Lines
src/download.jl 0.00% 2 Missing ⚠️
src/datasets/graphs/chickenpox.jl 0.00% 1 Missing ⚠️
src/datasets/graphs/ogbdataset.jl 50.00% 1 Missing ⚠️
src/datasets/graphs/temporalbrains.jl 0.00% 1 Missing ⚠️
src/datasets/graphs/traffic.jl 0.00% 1 Missing ⚠️
src/datasets/graphs/windmillenergy.jl 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #268      +/-   ##
==========================================
+ Coverage   45.94%   47.75%   +1.80%     
==========================================
  Files          50       54       +4     
  Lines        2501     2689     +188     
==========================================
+ Hits         1149     1284     +135     
- Misses       1352     1405      +53     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@CarloLucibello
CarloLucibello merged commit aa98e88 into JuliaML:master Oct 21, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants