Skip to content

fix: handle empty-feature depth slices in global rank - #372

Merged
johannesparty merged 2 commits into
mainfrom
fix/global-rank-empty-feature-slice
Jun 26, 2026
Merged

fix: handle empty-feature depth slices in global rank#372
johannesparty merged 2 commits into
mainfrom
fix/global-rank-empty-feature-slice

Conversation

@johannesparty

Copy link
Copy Markdown
Contributor

Problem

rank_soils_global crashes for some inputs with:

ValueError: Found array with 0 feature(s) (shape=(11, 0)) while a minimum of 1 is required by SimpleImputer.

(seen in production on tag 2026-06-24.1). The backend degrades gracefully (returns ALGORITHM_FAILURE), but the affected lookups get no ranking.

Cause. Per depth slice, rank_soils_global builds a feature matrix and passes it to gower_distances. When a slice has zero usable feature columns — the bedrock column-filter drops every property because the user's sample_pedon row is all-NaN at that depth — gower_distances feeds a shape=(n, 0) array into SimpleImputer, which raises.

Why now (regression). The crash line is old, but #368 (in 2026-06-24.1) changed user property arrays from a contiguous packed list to depth-indexed arrays that leave NaN at any depth the user didn't record. That makes all-NaN slices a normal outcome for gapped / non-zero-start horizon data (exactly the pedons #368 calls out), turning a latent crash into a reachable one.

Fix

  • Guard the per-slice gower call (now _slice_gower_distance): a 0-feature slice returns an all-NaN (n, n) matrix instead of calling gower_distances. The existing masked-average and NaN-infill steps already treat NaN distances as "no information", so the slice is ignored and components rank on the depths that do have data. An all-NaN matrix (rather than skipping) keeps the per-slice list aligned with soil_matrix rows.
  • Harden dis_max: max(map(np.nanmax, …))np.nanmax(…) so an all-NaN slice can't make the max NaN via max()'s ordering.

Tests

New soil_id/tests/test_global_rank_empty_slice.py (fast, no DB/network):

  • 0-feature slice → all-NaN (11, 11) matrix, no raise
  • characterization: the raw gower_distances path still raises on 0 features
  • non-empty slice passes through to gower_distances unchanged

Also verified the live test_global_integration path still ranks normally.

Scope

This is a crash/robustness fix only — it stops the failure and lets ranking proceed on available depths. It does not make global rank use data below 200 cm; that depth space is still hardcoded to 200 (min(bottom, 200), length-200 arrays) and is a separate, later change.

🤖 Generated with Claude Code

johannesparty and others added 2 commits June 26, 2026 11:10
rank_soils_global builds a per-depth-slice feature matrix and passes it to
gower_distances. When a depth slice has zero usable feature columns — e.g. the
user recorded a depth interval but left texture/rock-fragment/color blank, which
the bedrock column-filter can reduce to nothing — gower_distances fed a
shape=(n, 0) array into SimpleImputer, raising "Found array with 0 feature(s)…
minimum of 1 is required" and failing the whole ranking.

Guard the gower call: for a 0-feature slice emit an all-NaN (n, n) distance
matrix instead. The existing masked-average and NaN-infill steps already treat
NaN distances as "no information", so the slice is ignored and components are
ranked on the depths that do have data. An all-NaN matrix (rather than skipping
the slice) keeps dis_mat_list positionally aligned with soil_matrix rows.

Also make dis_max a single NaN-aware reduction over the stack so an all-NaN
slice can't turn the max NaN via max()'s ordering.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract the per-slice gower computation into _slice_gower_distance so the
empty-feature guard is unit-testable without a database or network, then add
fast regression tests:

- a 0-feature slice returns an all-NaN (n, n) matrix instead of crashing
- the raw gower path still raises on a 0-feature array (characterization, so
  the guard's rationale is revisited if that ever changes)
- a non-empty slice passes through to gower_distances unchanged

Behavior of rank_soils_global is unchanged; this only names the guard and
gives it a seam to test against.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@johannesparty
johannesparty requested a review from ssfrancis June 26, 2026 18:56
@ssfrancis

Copy link
Copy Markdown
Member

It would be fair to say that I reviewed lightly. Carissa or Garo might want to review as well if familiar with gower distance calculations.

@johannesparty
johannesparty merged commit a0d9ab2 into main Jun 26, 2026
4 checks passed
@johannesparty
johannesparty deleted the fix/global-rank-empty-feature-slice branch June 26, 2026 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants