Skip to content

Detect haploid / hemizygous input at load time#122

Merged
andrewkern merged 4 commits into
mainfrom
feature/haploid-detection
Jun 13, 2026
Merged

Detect haploid / hemizygous input at load time#122
andrewkern merged 4 commits into
mainfrom
feature/haploid-detection

Conversation

@andrewkern

@andrewkern andrewkern commented Jun 12, 2026

Copy link
Copy Markdown
Member

Summary

Makes pg_gpu's diploid-only loaders fail loudly on haploid and pseudo-diploid
input instead of silently producing doubled haplotype counts and biased
statistics. This is "layer 1" of #121 (the detection/guard layer); first-class
haploid support with an opt-in ploidy= / haploid_samples= parameter remains
as the follow-on.

Reported by @jiseonmin in #120: running a genome scan on the X chromosome with
a male-only VCF ran without warning but produced output that "assumes everyone
is homozygous everywhere." The assert ploidy == 2 in from_vcf never fired,
because read_vcf returns a ploidy-2 layout regardless of the VCF's actual
ploidy.

What changed

New check_diploid_encoding(gt, sample_names, source) (in pg_gpu/_warnings.py),
which inspects the actual allele values (not scikit-allel's padded shape) and:

  • Raises ValueError for a non-2 ploidy axis, or for any sample whose calls
    are genuinely haploid -- identified robustly as the [allele, -1] padding shape
    with no diploid evidence anywhere (a full [a, b] call or a reverse [-1, b]
    half-call, neither of which haploid padding can produce). This avoids
    misreading a partially-missing diploid call (GT=1/.) as haploid.
  • Warns HaploidDataWarning on the unambiguous global tell: a polymorphic
    dataset with zero heterozygous genotypes -- exactly the reporter's "everyone
    homozygous everywhere." A single real heterozygote anywhere suppresses it, so
    it stays quiet on ordinary diploid data where individuals are incidentally
    homozygous in a small region.

Wired into every loader path:

  • HaplotypeMatrix.from_vcf -- replaces the dead assert ploidy == 2
  • HaplotypeMatrix._build_eager (zarr) -- replaces the manual gt.shape[2] != 2 check
  • GenotypeMatrix.from_vcf and GenotypeMatrix._build_eager (zarr) -- previously unguarded
  • ZarrGenotypeSource.__init__ (streaming) -- O(1) ploidy-axis guard from store
    metadata, sharing one _require_diploid_ploidy helper with the value-based detector

Warnings consolidation

pg_gpu's warning classes had been accreting one-per-module
(MemoryLimitedWarning in _memory_warning.py, BadlyChunkedWarning in
streaming_matrix.py, and now this PR's HaploidDataWarning). Rather than add
a third scattered module, this PR consolidates all of them into a single
pg_gpu/_warnings.py:

  • The three warning classes live together at the top (the taxonomy), followed
    by the load-time helpers that emit them (the VCF size heuristic and the
    diploid-encoding check).
  • _memory_warning.py and _ploidy_check.py are deleted, folded into
    _warnings.py; BadlyChunkedWarning's class moves there too (its emit site
    stays in streaming_matrix, which now imports the class -- so
    streaming_matrix.BadlyChunkedWarning still resolves).
  • All three are exported from the package top-level, so every pg_gpu warning is
    silenceable via from pg_gpu import XxxWarning /
    warnings.filterwarnings(category=...).

The currently-untyped bare UserWarning call sites (in ld_statistics,
windowed_analysis, zarr_io, zarr_source) are intentionally left as-is.

What a user sees now

  • Pseudo-diploid male X (GT=1/1): loads, with a clear HaploidDataWarning
    naming the carriers and the bias it introduces.
  • True haploid male X (GT=1): ValueError naming the samples and explaining
    the half-missing-diploid problem.

Tests

New tests/test_ploidy_check.py (17 tests): unit coverage for normal diploid,
true-haploid (incl. mixed and sample naming), partial-missing and fully-missing
diploids not misread as haploid, monomorphic-reference no-warn,
heterozygote-suppresses-warn, ploidy 1/3/4; plus VCF integration tests through
both from_vcf loaders. The existing test_memory_warning.py repoints to the
consolidated module.

Full suite: 870 passed, 63 skipped. Lint clean.

Follow-on (not in this PR)

Layer 2 of #121: opt-in ploidy= / haploid_samples=, mixed-ploidy collapse,
and per-sample handling of pseudo-diploid stores on the streaming path.

Advances #121, addresses #120.

@andrewkern

Copy link
Copy Markdown
Member Author

@nspope this is a PR with the warnings layer

@andrewkern andrewkern requested a review from nspope June 12, 2026 23:17

@nspope nspope left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me ...

check_diploid_encoding(gt, source="test")


# --- integration through the VCF loaders -----------------------------------

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any need to test this through zarr loaders too? i.e. dump to vcz and reload?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call -- added in 1a22135. The detector is wired into the VCZ _build_eager path separately from from_vcf, so it warrants its own coverage. New zarr roundtrip tests:

  • pseudo-diploid HaplotypeMatrix dumped to VCZ via to_zarr and reloaded through both HaplotypeMatrix.from_zarr and GenotypeMatrix.from_zarr -> HaploidDataWarning
  • a normal diploid (heterozygous) store reloads quietly
  • a genuinely haploid store (ploidy-1 call_genotype axis) is rejected on load

@andrewkern andrewkern merged commit 78e0c49 into main Jun 13, 2026
1 check passed
@andrewkern andrewkern deleted the feature/haploid-detection branch June 13, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants