Skip to content

Add the SMOS preprocessing python scripts - #189

Draft
gmao-qliu wants to merge 4 commits into
developfrom
feature/qliu/smos_preproc
Draft

Add the SMOS preprocessing python scripts#189
gmao-qliu wants to merge 4 commits into
developfrom
feature/qliu/smos_preproc

Conversation

@gmao-qliu

Copy link
Copy Markdown
Contributor

Summary

Adds a new standalone pipeline that preprocesses SMOS L1C brightness temperature (Tb) data for near-real-time ingestion into MERRA21C-land:

EE .zip --(smos-ee-to-nc.sh)--> NetCDF --(preprocess_nc)--> EASEv2 M36 "REG" binaries --(SCLF1C_reg2fit)--> Tb40 "FIT" binaries

Normal operation is a daily cron job processing the previous day's data; the driver also supports on-demand backfill of a single day or an inclusive date range.

Requirements / external dependencies

  • Runs on Discover/NCCS under module load python/GEOSpyD (numpy, scipy, netCDF4, PyYAML).
  • Depends on an external, out-of-repo converter script (smos-ee-to-nc.sh, path set in config.yaml).
  • Depends on static, pre-generated regridding weights (data/GEOSIT_to_EASEv2_M36.mat) and an AUX Gal SM file — not committed to the repo; linked from production Discover paths.

Usage

module load python/GEOSpyD
python SMOSproc_main.py # NRT: process yesterday's data
python SMOSproc_main.py --date 20260801 # Backfill a single day
python SMOSproc_main.py --start 20260801 --end 20260803 # Backfill a date range

Notes for reviewers

  • This is a Python port (2021, Q. Liu) of prior MATLAB SCLF1C preprocessing code (G. De Lannoy); the science logic mirrors the original.
  • No automated tests are included; validation has been manual, comparing REG/FIT binary outputs against the legacy MATLAB pipeline.

Testing done

  • Ran end-to-end on Discover for sample backfill days and confirmed REG and FIT binaries are produced with expected file naming/paths for both ascending and descending passes.

@gmao-qliu gmao-qliu self-assigned this Aug 14, 2026
@gmao-qliu gmao-qliu added enhancement New feature or request 0-diff trivial very, very obvious 0-diff change pre-processing labels Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

0-diff trivial very, very obvious 0-diff change enhancement New feature or request pre-processing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant