Detect haploid / hemizygous input at load time#122
Merged
Conversation
Member
Author
|
@nspope this is a PR with the warnings layer |
nspope
approved these changes
Jun 13, 2026
| check_diploid_encoding(gt, source="test") | ||
|
|
||
|
|
||
| # --- integration through the VCF loaders ----------------------------------- |
Collaborator
There was a problem hiding this comment.
any need to test this through zarr loaders too? i.e. dump to vcz and reload?
Member
Author
There was a problem hiding this comment.
Good call -- added in 1a22135. The detector is wired into the VCZ _build_eager path separately from from_vcf, so it warrants its own coverage. New zarr roundtrip tests:
- pseudo-diploid
HaplotypeMatrixdumped to VCZ viato_zarrand reloaded through bothHaplotypeMatrix.from_zarrandGenotypeMatrix.from_zarr->HaploidDataWarning - a normal diploid (heterozygous) store reloads quietly
- a genuinely haploid store (ploidy-1
call_genotypeaxis) is rejected on load
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes pg_gpu's diploid-only loaders fail loudly on haploid and pseudo-diploid
input instead of silently producing doubled haplotype counts and biased
statistics. This is "layer 1" of #121 (the detection/guard layer); first-class
haploid support with an opt-in
ploidy=/haploid_samples=parameter remainsas the follow-on.
Reported by @jiseonmin in #120: running a genome scan on the X chromosome with
a male-only VCF ran without warning but produced output that "assumes everyone
is homozygous everywhere." The
assert ploidy == 2infrom_vcfnever fired,because
read_vcfreturns a ploidy-2 layout regardless of the VCF's actualploidy.
What changed
New
check_diploid_encoding(gt, sample_names, source)(inpg_gpu/_warnings.py),which inspects the actual allele values (not scikit-allel's padded shape) and:
ValueErrorfor a non-2 ploidy axis, or for any sample whose callsare genuinely haploid -- identified robustly as the
[allele, -1]padding shapewith no diploid evidence anywhere (a full
[a, b]call or a reverse[-1, b]half-call, neither of which haploid padding can produce). This avoids
misreading a partially-missing diploid call (
GT=1/.) as haploid.HaploidDataWarningon the unambiguous global tell: a polymorphicdataset with zero heterozygous genotypes -- exactly the reporter's "everyone
homozygous everywhere." A single real heterozygote anywhere suppresses it, so
it stays quiet on ordinary diploid data where individuals are incidentally
homozygous in a small region.
Wired into every loader path:
HaplotypeMatrix.from_vcf-- replaces the deadassert ploidy == 2HaplotypeMatrix._build_eager(zarr) -- replaces the manualgt.shape[2] != 2checkGenotypeMatrix.from_vcfandGenotypeMatrix._build_eager(zarr) -- previously unguardedZarrGenotypeSource.__init__(streaming) -- O(1) ploidy-axis guard from storemetadata, sharing one
_require_diploid_ploidyhelper with the value-based detectorWarnings consolidation
pg_gpu's warning classes had been accreting one-per-module
(
MemoryLimitedWarningin_memory_warning.py,BadlyChunkedWarninginstreaming_matrix.py, and now this PR'sHaploidDataWarning). Rather than adda third scattered module, this PR consolidates all of them into a single
pg_gpu/_warnings.py:by the load-time helpers that emit them (the VCF size heuristic and the
diploid-encoding check).
_memory_warning.pyand_ploidy_check.pyare deleted, folded into_warnings.py;BadlyChunkedWarning's class moves there too (its emit sitestays in
streaming_matrix, which now imports the class -- sostreaming_matrix.BadlyChunkedWarningstill resolves).silenceable via
from pg_gpu import XxxWarning/warnings.filterwarnings(category=...).The currently-untyped bare
UserWarningcall sites (inld_statistics,windowed_analysis,zarr_io,zarr_source) are intentionally left as-is.What a user sees now
GT=1/1): loads, with a clearHaploidDataWarningnaming the carriers and the bias it introduces.
GT=1):ValueErrornaming the samples and explainingthe half-missing-diploid problem.
Tests
New
tests/test_ploidy_check.py(17 tests): unit coverage for normal diploid,true-haploid (incl. mixed and sample naming), partial-missing and fully-missing
diploids not misread as haploid, monomorphic-reference no-warn,
heterozygote-suppresses-warn, ploidy 1/3/4; plus VCF integration tests through
both
from_vcfloaders. The existingtest_memory_warning.pyrepoints to theconsolidated module.
Full suite: 870 passed, 63 skipped. Lint clean.
Follow-on (not in this PR)
Layer 2 of #121: opt-in
ploidy=/haploid_samples=, mixed-ploidy collapse,and per-sample handling of pseudo-diploid stores on the streaming path.
Advances #121, addresses #120.