NVIDIA/earth2studio 0.20.0

v0.20.0

Repository: NVIDIA/earth2studio

Tag: 0.20.0

Published: 2026-10-29T23:54:21Z

Prerelease: no

Release notes: Earth2Studio general release 0.20.0

:rotating_light: Breaking Changes :rotating_light:

• NCEP observation frames now use Arrow-backed dtypes throughout. Missing values are nulls, string columns are dictionary-encoded, and NNJAObsSatwnd quality indicators use list> instead of Python lists. Consumers that depend on the previous pandas dtypes or missing-value representation should update accordingly.
• CorrDiffEra5Hrrr.load_model now defaults to the distilled model; pass distilled_model=False to use the 50-step teacher.

:closed_book: Changelog

:earth_americas: Added

• Added subseasonal-to-seasonal evaluation capability in the eval recipe, with WindowMeanForecastPipeline and WindowMeanSource.
• Added the distilled ERA5 -> HRRR downscaling model (CorrDiffEra5HrrrDistilled).
• Added StormScope Flash variants of StormScopeGOES and StormScopeMRMS for accelerated inference over CONUS, including crop-based inference.
• Added HealDA v2 (HealDAv2), a global machine-learning data-assimilation model mapping a 48-hour sequence of satellite and conventional observations to a 104-channel, 0.25° atmospheric analysis (da-healda-v2 extra). It accepts one wide NNJA frame (layout="wide") per stream in input_coords() order.
• Added InterpCRPSDiT, a CRPS DiT temporal interpolation model with sub-hourly output.
• Added an original_event option and report time/latitude/longitude columns to NNJAObsConv; report-time filtering preserves complete PrepBUFR profiles.
• Added a detector column to NNJAObsSat carrying the CrIS field-of-view index, plus scan_quality, granule_quality, and footprint_quality columns for ATMS, CrIS, and IASI provider flags.
• Added a cycle_time column to the NCEP observation sources (NNJAObsConv, NNJAObsSatwnd, NNJAObsSat, NomadsGDASObsConv) identifying the cycle of the file each row was decoded from.
• Added a layout option to the NCEP observation sources: "wide" returns one row per report level, wind, or footprint with one observation_ column per variable; "long" remains the default.
• Added fetch_dataframes to fetch several DataFrame requests concurrently, accepting and returning either a list or a dictionary.
• Added the Google Cloud mirror (source="google") to ECMWF open-data sources.
• Eval recipe: scoring.online.verification_sources scores one online rollout against several named truth sources in a single pass, writing separate statistics and scores per reference.

⚙️ Changed

• The NCEP observation sources decode BUFR with earth2bufr instead of PyBufrKit. Rows and values are unchanged except PrepBUFR station identifiers with non-ASCII bytes, which now carry spaces instead of U+FFFD.
• NCEP observation frames use pd.ArrowDtype columns throughout, with null missing values and dictionary-encoded string columns.
• NNJAObsSat returns a row for every decoded channel, including channels with missing or unconvertible observations, so every footprint appears.
• NNJAObsSatwnd reads each field from its first occurrence in the message template.
• NNJAObsSat decodes only the sensor_indices channels of an IR sounder and returns IASI and CrIS brightness temperature from a single Planck conversion.
• decode_workers on NCEP sources controls the threads transforming decoded batches; decoding runs on earth2bufr's thread pool.
• NNJA and GDAS conventional lexicon modifiers are now identity operations; the decoders return Earth2Studio units.
• fetch_dataframe preserves attributes set by the data source and adds request_time and request_lead_time instead of replacing them.
• obstore_read_chunked and obstore_fetch_to_cache accept chunk_size, max_concurrent, and hedge_after. NNJA sources use larger concurrent ranges and re-request stalled ranges.
• NNJAObsConv and NNJAObsSat expose an adjustable task_timeout, defaulting to 600 seconds per download attempt.
• ECMWF open-data sources accept a client_kwargs dictionary for opendata.Client.
• Updated CBottle Tropical Cyclone and Infill checkpoints to new EMA versions.
• Added rollout checkpointing support for AtlasCRPS.
• Reduced the default HealDA v2 example observation downloads and extended its download timeouts.

:hammer: Fixed

• NNJAObsConv and NomadsGDASObsConv correctly decode PrepBUFR files from before 2007 and raise on files with no known message family instead of returning no rows.
• PrepBUFR messages dated 2000 (edition 3, year of century 100) get the year 2000 instead of 100, preventing their observations from being dropped by the time window.
• NNJAObsConv, NNJAObsSatwnd, and NNJAObsSat raise on file, message, or DX-table decode failures instead of silently returning fewer rows.
• NNJAObsSat skips sensors outside their archive years in multi-sensor requests instead of failing the whole request.
• NNJA, GDAS, and UFS observation sources read all cycle files whose ±3-hour spans overlap a request and keep only rows within each requested window.
• Updated the CorrDiffCosmoEra5 checkpoint revision to use corrected ERA5 training-domain statistics in rea2/stats.json.
• Brier scores honor custom ensemble dimension names during coordinate validation.
• Corrected the Energy Score ensemble reduction axis when multivariate dimensions precede the ensemble dimension.
• Fixed running variance and standard deviation when reducing non-leading or multiple dimensions.
• Aurora1p5 and Aurora1p5Ensemble no longer apply a second log-untransform to tp1h and sf1h, which made hourly precipitation and snowfall about 1000 times too small.
• Fixed ZarrBackend and IceChunkBackend reopening stores containing root-level subgroups or scalar arrays.
• datasource_to_file offsets requested times by each lead time independently instead of compounding the leads.
• DataArrayPathList preserves size-one dimensions when loading files with a single variable or time.
• Added the missing data extra to the HealDA v2 example dependencies.

📦 Dependencies

• Replaced PyBufrKit with earth2bufr in the data extra.
• Require pyarrow>=24 to avoid Table.filter corruption on large single chunks in versions 18–23.
• Switched to stable PyPI ONNX Runtime with CUDA 13 support.
• Updated pinned requirements to aiohttp==3.14.5, fsspec==2026.9.0, and urllib3==2.8.0.

Thank you to our contributors:

-...