Skip to content

fix(#203): landfire STAC no longer advertises a fractions asset that was purged on purpose - #655

Open
cboettig wants to merge 1 commit into
mainfrom
fix/landfire-hexdesc-dangling
Open

fix(#203): landfire STAC no longer advertises a fractions asset that was purged on purpose#655
cboettig wants to merge 1 commit into
mainfrom
fix/landfire-hexdesc-dangling

Conversation

@cboettig

@cboettig cboettig commented Sep 5, 2026

Copy link
Copy Markdown
Member

Fixes the dangling fractions reference found while triaging #203.

The defect

Both published LANDFIRE collections carried this on their mode hex asset:

"Each cell takes the class covering the largest share of it, so the mix within a cell is not preserved; for the area held by each class use the fractional-coverage asset instead."

There is no such asset, and there was never meant to be one. fractions was dropped deliberately for this source, with measurements recorded in BUILD.md:

Measured against the exact source pixel histogram, mode reproduces the true CONUS VCC distribution to within 0.72 percentage points […] a cell holds ~17 pixels and VCC stands are far larger than 15,000 m², so most cells are effectively pure. […] The two partial hex-fractions prefixes (vcc 1/6, evt 2/6, ~9 GB) were purged so they cannot fake coverage.

make-stac.py already had a fractions_published flag — but it only popped the asset and rewrote the collection-level description. hex_desc was built unconditionally, so the asset-level sentence survived and pointed at something purged on purpose. verify-stac.py does not catch it: the reference is in prose, not an entry in assets.

Two fixes

1. hex_desc branches on fractions_published. The no-fractions text states the limit and names the reducer that answers it, rather than pointing at nothing:

"…so the mix within a cell is not preserved, and no per-class fractional-coverage asset is published for this layer. At a 30 m source and resolution 10 a cell holds roughly 17 pixels, so where stands are large relative to the cell the dominant class closely tracks the true class distribution; for absolute acreage, rare or interspersed classes, and ecotone work, a fractional-coverage build is the reducer that answers it."

No layer-specific figure in the string — the 0.72 pp measurement is VCC's and would be false in EVT's description.

2. bucket_collection() had the same class of drift, in the other direction. It claimed all four products "in both dominant-class and per-class fractional-coverage form" and emitted child links for all four, where only vcc and evt are published. The live bucket collection is correct because it was corrected by hand at publish time — so re-running the committed generator would have regressed it to the false claim and added two 404 child links. Added a PUBLISHED set and matched the live text.

Published and verified

Republished landfire-2024-vcc and landfire-2024-evt through a geo-workflows job (no local NRP credentials), patching only assets.<layer>-hex.description:

check result
collateral changes none — asserted the docs are identical with that one field masked
pre-publish verify-stac.py --no-data PASS, both
upload readback byte-identical to the verified local file, both
post-publish verify-stac.py --bucket public-landfire --dataset landfire-2024-vcc exit 0
post-publish … --dataset landfire-2024-evt exit 0
dangling sentence present on live S3 False, both

Code and data agree: stubbing the legend/histogram readers with the published values and running build() reproduces the live description byte-for-byte for both layers, and bucket_collection() now reproduces the live bucket document exactly (description and child list). So the next generator run is a no-op rather than a regression.

The bucket collection itself needed no republish — it was already correct on S3. This only stops the generator from undoing it.

Not addressed here

@cboettig
fix(#203): landfire STAC no longer advertises a fractions asset that …
6ab2ed3 
…was purged on purpose

Both published collections told consumers, on the mode hex asset, "for the area
held by each class use the fractional-coverage asset instead". No such asset
exists: fractions were dropped deliberately for this source (BUILD.md - at 30 m
and res 10 a cell holds ~17 pixels and stands are far larger, so mode tracked the
true CONUS VCC distribution to within 0.72 pp), and the two partial prefixes
(vcc 1/6, evt 2/6, ~9 GB) were purged so they could not fake coverage.

make-stac.py already had a `fractions_published` flag, but it only popped the
asset and rewrote the collection-level description; `hex_desc` was built
unconditionally, so the asset-level sentence survived. verify-stac.py does not
catch it because the reference is prose, not an entry in `assets`.

- hex_desc now branches on fractions_published. The no-fractions text states the
  limit and names the reducer that answers it, instead of pointing at nothing.
- bucket_collection() had the same class of drift in the other direction: it
  claimed all four products "in both dominant-class and per-class
  fractional-coverage form" and emitted child links for all four. Only vcc and
  evt are published. That was corrected by hand at publish time, so re-running
  the generator would have regressed the live document back to the false claim
  and added two 404 child links. Added PUBLISHED and matched the live text.

Republished landfire-2024-vcc and landfire-2024-evt via a geo-workflows job,
patching only assets.<layer>-hex.description (asserted no other key changed).
verify-stac.py --bucket public-landfire exits 0 for both, and build() now
reproduces the published description byte-for-byte for both layers.

The bucket collection needed no republish - it was already correct on S3; this
only stops the generator from undoing it.
@cboettig cboettig mentioned this pull request Sep 5, 2026
Open
9 tasks
cboettig added a commit that referenced this pull request Sep 14, 2026
@cboettig
feat(#515): STAC for the seven LANDFIRE structure/fuels collections
3d470d7 
make-stac-structure.py authors all seven and the public-landfire bucket collection.
Two shapes in one tranche needing opposite treatment:

  CONTINUOUS (cbd, cbh, cc, ch) -- linearly scaled physical quantities. `mean` reducer,
  decode from the shipped CSV (KGM3 = VALUE/100, METERS = VALUE/10, CC_PERCENT = VALUE),
  and NO classification:classes: a continuous surface has no class list, and #628's rule
  is that anything listed there gets painted.

  CATEGORICAL (fbfm13, fvc, fvh) -- class codes. `mode` reducer, classification:classes
  from the shipped legend with fill codes REMOVED per #628.

Every range, mean and `values` array is MEASURED from the published hex rather than taken
from upstream documentation (#518). Verified: all 48 fvc / 40 fvh / 17 fbfm13 values
present in the data are legend members, and no fill code appears in any class list.

The four facts a consumer cannot recover from the data are all in the published text:
  - the decode factor -- without it a reader takes 12 for 12 kg/m^3 rather than 0.12;
  - `0 = Non-Forested` is a category, not a measurement, and is ~69% of valid pixels, so
    it is excluded and a mean over these layers is a mean over FOREST, not over all land;
  - cbh's top class (stored 100) means ">= 10 m" and holds ~19% of cells, so its mean is
    pulled toward that ceiling;
  - cc has no class below 10% cover -- sparse tree cover falls into 0 and is absent, so
    the layer cannot separate open woodland from treeless ground.

One prose bug the gate could not catch: cc's decode sentence read "the stored value is
not the physical quantity: the stored value is already a percentage to obtain percent",
because the template assumed a scale factor and cc's is identity. verify-stac.py passed
it -- it checks structure, not whether the sentence parses. Now branches on scale == 1.0.

Bucket collection gets a SINGLE OWNER. make-stac.py's bucket_collection() knows only the
original four layers, so running it against public-landfire/stac-collection.json would
silently drop these seven; it is marked superseded in a docstring and this file carries
the full nine-product roster. Two generators for one file is how the fractions claim
drifted the first time (PR #655).

Gates, all green:
  pre-publish  verify-stac.py --no-data              PASS x7 + bucket
  upload       readback byte-identical               x7 + bucket
  post-cluster verify-stac.py --bucket --dataset     exit 0 x7
               (includes the data-backed values == ingested DISTINCT check via the MCP)
  bucket collection 2 -> 9 children; public-landfire already registered in the root.
@cassiebuhler cassiebuhler mentioned this pull request Sep 16, 2026
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@cboettig