arXiv Statistics ML

The Ensemble Kalman Inversion Race

The paper compares different Ensemble Kalman methods for calibrating climate model parameters by minimizing the misfit between modeled and observed climate statistics. It conducts systematic numerical experiments on Lorenz-type models, including neural network parameterizations, to evaluate computational efficiency and accuracy of each method. The study examines how prior information and dimensionality affect the cost of these methods.

arXiv Machine Learning
Jun 26

Huracan: A skillful end-to-end data-driven system for ensemble data assimilation and weather prediction

arXiv:2508. 18486v2 Announce Type: replace-cross Abstract: Over the past few years, machine learning-based data-driven weather prediction has been transforming operational weather forecasting by providing more accurate forecasts while using a mere fraction of computing power compared to traditional numerical weather prediction (NWP).

By Zekun Ni, Jonathan Weyn, Hang Zhang, Yanfei Xiang, Jiang Bian, Weixin Jin, Kit Thambiratnam, Qi Zhang, Haiyu Dong, Hongyu Sun
arXiv Machine Learning
Oct 28

Probabilistic adjustment of seasonal forecasts using cVAE with a non-parametric decoder: A case study of Arctic sea ice predictions

The paper presents a probabilistic post‑processing framework that uses a conditional Variational Autoencoder (cVAE) with a non‑parametric decoder to adjust seasonal Arctic sea‑ice forecasts. By replacing the Gaussian decoder with a generator and optimizing with Continuous Ranked Probability Score (CRPS), the method produces larger, bias‑corrected ensembles that are better calibrated, more consistent with observations, and exhibit higher resolution and sharper spectral power than standard cVAEs. The approach demonstrates improved forecast skill and reduced errors compared to benchmark predictions.

By Parsa Gooya, Reinel Sospedra-Alfonso
arXiv Statistics ML
Oct 26

Batched NN 4D-Var: a performant hybrid model framework for the Integrated Forecasting System

Hybrid forecasting systems that combine machine learning with numerical weather prediction are explored using a neural‑network representation of model error within the Integrated Forecasting System. The study introduces a targeted neural‑network 4D‑Var approach that estimates only network parameters and a batched extension that processes assimilation windows in random order, averaging parameter updates across multiple windows. Experiments show that while sequential online learning improves forecasts during training, it fails to generalise, whereas the batched formulation consistently enhances out‑of‑sample forecast skill and improves convergence with larger batch sizes.

By Alban Farchi, Marcin Chrust, Massimo Bonavita, Marc Bocquet, Wei Pan, Patrick Laloyaux
arXiv Machine Learning
1d ago

Auto-differentiable data assimilation: Co-learning of states, dynamics, and filtering algorithms

The paper presents a framework called auto‑differentiable filtering that jointly learns the state, dynamics, and filtering algorithm parameters in data assimilation. It uses a loss function that allows gradient‑based optimization from partial, noisy observations via auto‑differentiation. Experiments demonstrate the method on diverse dynamical systems, including aerospace, atmospheric, biological, and geophysical models, and the authors offer guidelines for tailoring the framework to specific observation models, accuracy needs, and computational budgets.

By Melissa Adrian, Daniel Sanz-Alonso, Rebecca Willett
arXiv Machine Learning
Oct 8

Artificial intelligence pathways from weather to climate

arXiv:2610.09770v1 Announce Type: cross Abstract: Deep learning has made rapid advances in weather forecasting: autoregressive models trained on atmospheric reanalyses now rival dynamical models acro...

By Tom Beucler, J. David Neelin, Hui Su, Shivanshi Asthana, Chris Bretherton, Will Chapman, Costa Christopoulos, Spencer K. Clark, Aditya Grover, Ignacio Lopez-Gomez, Tapio Schneider, Adam Subel, Oliver Watt-Meyer
arXiv Statistics ML
Sep 17

A Continuous-Time Ensemble Kalman-Bucy Smoother for Causal Inference and Model Discovery

The paper presents an ensemble Kalman–Bucy smoother (EnKBS) for continuous‑time data assimilation of nonlinear dynamical systems, reconstructing conditional distributions from ensemble moments without needing tangent‑linear or adjoint models. It demonstrates that EnKBS achieves exact smoothing mean and covariance in the infinite‑ensemble limit for linear‑Gaussian systems and incorporates regularization techniques like covariance localization and inflation for high‑dimensional problems. The method is applied to Bayesian inference of causal relationships in a dyadic trigger‑feedback model and to an iterative learning algorithm that uncovers the structure and hidden parameters of a reduced‑order model of midlatitude atmospheric circulation, all with small ensembles under partial observations.

By Zhang Jiang (University of Wisconsin-Madison), Marios Andreou (University of Wisconsin-Madison), Sebastian Reich (University of Potsdam), Nan Chen (University of Wisconsin-Madison)