Skip to content

Commit 4db9d81

Browse files
WardDebWardDeb
authored
4.0.0 cleanup (#1450)
* update changelog * mend * docs orphan for warnings * drop doc reqs, fix python docstring * drop unused code from new entrypoints * point rtd to docs req * drop mpl from action reqs * action to test docs * minimal versions, include lock * centralize version check deeptools/galaxy, rust version in docs/deeptools * include authors * ensure rust in rtd yaml is an allowed rust version * cleanup readme, https * docs overhauls, remove plotly refs, update parser * rust 1.87 minimum * ignore venv * move pytest action over to pixi * pip as a pixi dep * of course dist upload needs mussllinux builds * set stack min for osx build in pixi env * up stack * 100MB stack for osx build ? * strip wheels, cap rust version to avoid conda-forge rust build error for osx * https in docs, clarify installation instructions, delete rst
1 parent 821c4a1 commit 4db9d81

65 files changed

Lines changed: 11929 additions & 2449 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/pypi.yml‎

Lines changed: 0 additions & 38 deletions
This file was deleted.

‎.github/workflows/test_docs.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
name: tests_docs
2+
on: [push, pull_request, workflow_dispatch]
3+
4+
permissions:
5+
contents: read
6+
7+
jobs:
8+
build-docs:
9+
name: Build Sphinx documentation
10+
runs-on: ubuntu-latest
11+
steps:
12+
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
13+
with:
14+
persist-credentials: false
15+
- uses: prefix-dev/setup-pixi@d3f436a425481402e6a95a1d1fc10331c708cd9e # v0.10.2
16+
with:
17+
environments: default
18+
- name: build docs
19+
run: pixi run docs

‎.github/workflows/test_pytest.yml‎

Lines changed: 20 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -4,108 +4,58 @@ on: [push, pull_request, workflow_dispatch]
44
permissions:
55
contents: read
66

7-
defaults:
8-
run:
9-
shell: bash -l {0}
10-
117
jobs:
128
check_versions_matches:
13-
name: Check deeptools version matches galaxy tools
9+
name: Check deeptools/galaxy/Rust versions stay in sync
1410
runs-on: ubuntu-latest
15-
if: github.base_ref == 'master'
1611
steps:
1712
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
1813
with:
1914
persist-credentials: false
20-
- name: Check path
21-
run: find /home/runner/work/deepTools/deepTools -name "pyproject.toml"
22-
- name: Get Version of Deeptools
23-
run: echo "deeptools_version=$(grep "version" /home/runner/work/deepTools/deepTools/pyproject.toml | awk '{print substr($NF, 2, length($NF) - 2)}')" >> $GITHUB_ENV
24-
- name: Get Version of Galaxy tools
25-
run: echo "galaxy_deeptools_version=$(grep "token.*TOOL_VERSION" /home/runner/work/deepTools/deepTools/galaxy/wrapper/deepTools_macros.xml | awk -F '>|<' '{print $3}')" >> $GITHUB_ENV
26-
- name: Versions
27-
run: |
28-
echo deeptools version=${deeptools_version}
29-
echo deeptools version in galaxy=${galaxy_deeptools_version}
30-
- name: Fail PR if versions not matching
31-
run: |
32-
if [ -z $galaxy_deeptools_version ]; then
33-
echo "galaxy_deeptools_version not defined change the github action"
34-
exit 1
35-
fi
36-
if [ -z $deeptools_version ]; then
37-
echo "deeptools_version not defined change the github action"
38-
exit 1
39-
fi
40-
if [ "$galaxy_deeptools_version" != "$deeptools_version" ]; then
41-
echo "versions do not match"
42-
exit 1
43-
fi
15+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
16+
with:
17+
python-version: "3.x"
18+
- name: Check version sync
19+
run: python3 scripts/check_version_sync.py
4420
build-linux:
4521
name: Test on Linux
4622
runs-on: ubuntu-latest
4723
strategy:
4824
matrix:
49-
python-version: ["3.12", "3.13", "3.14"]
25+
pixi-env: [py312, py313, py314]
5026
steps:
5127
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
5228
with:
5329
persist-credentials: false
54-
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
30+
- uses: prefix-dev/setup-pixi@d3f436a425481402e6a95a1d1fc10331c708cd9e # v0.10.2
5531
with:
56-
python-version: ${{ matrix.python-version }}
57-
cache: "pip"
58-
- name: Install Rust
59-
uses: actions-rust-lang/setup-rust-toolchain@166cdcfd11aee3cb47222f9ddb555ce30ddb9659 # v1.17.0
32+
environments: ${{ matrix.pixi-env }}
6033
- name: build deeptools
61-
run: |
62-
python -m venv venv
63-
source venv/bin/activate
64-
pip install maturin
65-
maturin develop --release --extras actions
34+
run: pixi run -e ${{ matrix.pixi-env }} build
6635
- name: test deeptools
67-
run: |
68-
source venv/bin/activate
69-
pytest -v
36+
run: pixi run -e ${{ matrix.pixi-env }} test -v
7037
- name: build wheel
71-
run: |
72-
source venv/bin/activate
73-
python -m build
38+
run: pixi run -e ${{ matrix.pixi-env }} python -m build
7439
- name: test wheel
75-
run: |
76-
source venv/bin/activate
77-
pip install dist/*whl
40+
run: pixi run -e ${{ matrix.pixi-env }} bash -c "pip install dist/*.whl"
7841
build-osx:
7942
name: Test on OSX
8043
runs-on: macOS-latest
8144
strategy:
8245
matrix:
83-
python-version: ["3.12", "3.13", "3.14"]
46+
pixi-env: [py312, py313, py314]
8447
steps:
8548
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
8649
with:
8750
persist-credentials: false
88-
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
51+
- uses: prefix-dev/setup-pixi@d3f436a425481402e6a95a1d1fc10331c708cd9e # v0.10.2
8952
with:
90-
python-version: ${{ matrix.python-version }}
91-
cache: "pip"
92-
- name: Install Rust
93-
uses: actions-rust-lang/setup-rust-toolchain@166cdcfd11aee3cb47222f9ddb555ce30ddb9659 # v1.17.0
53+
environments: ${{ matrix.pixi-env }}
9454
- name: build deeptools
95-
run: |
96-
python -m venv venv
97-
source venv/bin/activate
98-
pip install maturin
99-
maturin develop --release --extras actions
55+
run: pixi run -e ${{ matrix.pixi-env }} build
10056
- name: test deeptools
101-
run: |
102-
source venv/bin/activate
103-
pytest -v
57+
run: pixi run -e ${{ matrix.pixi-env }} test -v
10458
- name: build wheel
105-
run: |
106-
source venv/bin/activate
107-
python -m build
59+
run: pixi run -e ${{ matrix.pixi-env }} python -m build
10860
- name: test wheel
109-
run: |
110-
source venv/bin/activate
111-
pip install dist/*whl
61+
run: pixi run -e ${{ matrix.pixi-env }} bash -c "pip install dist/*.whl"

‎.github/workflows/wheels.yml‎

Lines changed: 2 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,4 @@
1-
name: build wheels
2-
# maturin generate-ci github > .github/workflows/maturin.yml
3-
# Also, see: https://github.com/jackh726/bigtools/blob/master/.github/workflows/release-python.yml
1+
name: build_wheels
42

53
on:
64
push:
@@ -103,41 +101,6 @@ jobs:
103101
name: wheels-musllinux-${{ matrix.platform.target }}
104102
path: dist
105103

106-
# windows:
107-
# runs-on: ${{ matrix.platform.runner }}
108-
# strategy:
109-
# matrix:
110-
# platform:
111-
# - runner: windows-latest
112-
# target: x64
113-
# - runner: windows-latest
114-
# target: x86
115-
# steps:
116-
# - uses: actions/checkout@v4
117-
# - uses: actions/setup-python@v5
118-
# with:
119-
# python-version: 3.x
120-
# architecture: ${{ matrix.platform.target }}
121-
# - name: Install And Configure Dependencies
122-
# env:
123-
# VCPKG_DEFAULT_TRIPLET: ${{ matrix.platform.target }}-windows
124-
# OPENSSL_DIR: ${{ github.workspace }}/vcpkg/installed/${{ matrix.platform.target }}-windows
125-
# OPENSSL_ROOT: ${{ github.workspace }}/vcpkg/installed/${{ matrix.platform.target }}-windows
126-
# LZMA_ROOT: ${{ github.workspace }}/vcpkg/installed/${{ matrix.platform.target }}-windows
127-
# run: |
128-
# vcpkg install openssl liblzma
129-
# - name: Build wheels
130-
# uses: PyO3/maturin-action@v1
131-
# with:
132-
# target: ${{ matrix.platform.target }}
133-
# args: --release --out dist --find-interpreter
134-
# # sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
135-
# - name: Upload wheels
136-
# uses: actions/upload-artifact@v4
137-
# with:
138-
# name: wheels-windows-${{ matrix.platform.target }}
139-
# path: dist
140-
141104
macos:
142105
runs-on: ${{ matrix.platform.runner }}
143106
strategy:
@@ -159,7 +122,6 @@ jobs:
159122
with:
160123
target: ${{ matrix.platform.target }}
161124
args: --release --out dist --find-interpreter
162-
# sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
163125
- name: Upload wheels
164126
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
165127
with:
@@ -177,7 +139,6 @@ jobs:
177139
with:
178140
command: sdist
179141
args: --out dist
180-
# sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
181142
- name: Upload sdist
182143
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
183144
with:
@@ -188,7 +149,7 @@ jobs:
188149
name: Release
189150
runs-on: ubuntu-latest
190151
if: ${{ startsWith(github.ref, 'refs/tags/') || github.event_name == 'workflow_dispatch' }}
191-
needs: [linux, macos, sdist]
152+
needs: [linux, musllinux, macos, sdist]
192153
permissions:
193154
id-token: write
194155
contents: write

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,6 @@ tool_test*
5959
env.yml
6060

6161
# Rust stuff
62-
Cargo.lock
6362
target*
6463

6564
# snakemake
@@ -73,3 +72,5 @@ target*
7372

7473
# pixi
7574
.pixi*
75+
76+
.venv

‎.pre-commit-config.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,11 @@ repos:
33
rev: v1.11.0
44
hooks:
55
- id: zizmor
6+
- repo: local
7+
hooks:
8+
- id: check-version-sync
9+
name: check Rust versions (docs - toml) and deeptools versions (toml - galaxy) to stay in sync
10+
entry: python3 scripts/check_version_sync.py
11+
language: system
12+
files: ^(Cargo\.toml|pyproject\.toml|\.readthedocs\.yaml|galaxy/wrapper/deepTools_macros\.xml)$
13+
pass_filenames: false

‎.readthedocs.yaml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@ version: 2
33
build:
44
os: ubuntu-22.04
55
tools:
6-
python: "3.12"
7-
rust: "1.82"
6+
python: "3.14"
7+
rust: "1.91"
88
apt_packages:
99
- clang
1010

@@ -15,4 +15,5 @@ python:
1515
install:
1616
- method: pip
1717
path: .
18-
- requirements: docs/requirements.txt
18+
extra_requirements:
19+
- docs

‎CHANGES.txt‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,30 @@
1+
4.0.0
2+
* new Rust-backed core for bamCoverage, bamCompare, computeMatrix, alignmentSieve and multiBamSummary (rayon multithreading, bigtools-based bigWig I/O), replacing the pure-Python implementations for these five tools; the previous Python implementations remain available as bamCoverage_old, bamCompare_old, computeMatrix_old, alignmentSieve_old and multiBamSummary_old during the transition, but will be removed in a future release
3+
* gzipped GTF/BED region files are now supported in multiBamSummary, computeMatrix and alignmentSieve; gzipped blacklist files are supported in all five Rust-backed tools (multiBamSummary, computeMatrix, alignmentSieve, bamCompare, bamCoverage)
4+
* --exactScaling is removed as this is the only scaling option available with the new backend
5+
* --ignoreDuplicates has been removed from the bamCoverage and bamCompare tools. Duplicate reads can be removed via the --samFlagExclude option (assuming the provided BAM file has duplicate reads marked)
6+
* plotPCA re-implemented with a lightweight scipy/numpy SVD backend. Proper handling of --transpose, --log2/--rowCenter and --ntop options.
7+
* plotly backend removed entirely; plotCorrelation, plotPCA, plotHeatmap, plotProfile and plotEnrichment now use matplotlib, with a new optional --ggplot theme, and computeGCBias's --plotFileFormat plotly option has also been removed
8+
* removed the standalone cm.py colormap module; colormap handling now lives in the plotting tools directly
9+
* plot labels now show sample names only per default (dropped .filtered.bam/.bed suffixes)
10+
* prebuilt wheels now cover Linux manylinux_2_28 and musllinux_1_2 (x86_64 + aarch64) and macOS (Intel + Apple Silicon), built via maturin
11+
* CI overhauled: pytest/rust/planemo test workflows split up (test_pytest.yml, test_rust.yml, test_planemo.yml), zizmor hardening for GitHub Actions, dependabot enabled
12+
* extensive new pytest/cargo test coverage across the Rust-backed tools and plotCorrelation/plotPCA/plotFingerprint data outputs
13+
* --nanAfterEnd doesn't rescale the inner region anymore in reference-point mode
14+
* BED output doesn't show wrong blockstart values anymore
15+
* blacklist filtering is done at bp level instead of genome chunk level for rewritten tools
16+
* alignmentSieve output order matches input order exactly
17+
* --missingDataAsZero no longer takes bases exceeding chromosome bounds as 0 values but rather purges the bins
18+
* large scale values precision slightly altered with new backend (f32 vs f64)
19+
20+
3.5.6
21+
* minimal supported python version raised to 3.9 (numpy >= 2 support); NaN handling switched to np.nan
22+
* drop conda usage in pypi/test/planemo CI in favor of pip installs; samtools now pulled from conda-forge/bioconda where still needed in CI
23+
* include GRCm39 in the effective genome size docs
24+
* add ARM/osx specific installation instructions
25+
* computeMatrix: ensure --beforeRegionStartLength/-b and --afterRegionStartLength/-a are always treated as positive values
26+
* minor --version output fixes in plotCoverage and multiBigwigSummary
27+
128
3.5.5
229
* drop support for python 3.7
330
* doc fixes (argparse properly displayed, minor changes in installation instructions)

0 commit comments

Comments
 (0)