The interface of data assimilation and machine learning
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
The Flow has not summarised this story yet — read it at arXiv Statistics ML.
arXiv:2508. 18486v2 Announce Type: replace-cross Abstract: Over the past few years, machine learning-based data-driven weather prediction has been transforming operational weather forecasting by providing more accurate forecasts while using a mere fraction of computing power compared to traditional numerical weather prediction (NWP).
The paper presents a framework called auto‑differentiable filtering that jointly learns the state, dynamics, and filtering algorithm parameters in data assimilation. It uses a loss function that allows gradient‑based optimization from partial, noisy observations via auto‑differentiation. Experiments demonstrate the method on diverse dynamical systems, including aerospace, atmospheric, biological, and geophysical models, and the authors offer guidelines for tailoring the framework to specific observation models, accuracy needs, and computational budgets.
Hybrid forecasting systems that combine machine learning with numerical weather prediction are explored using a neural‑network representation of model error within the Integrated Forecasting System. The study introduces a targeted neural‑network 4D‑Var approach that estimates only network parameters and a batched extension that processes assimilation windows in random order, averaging parameter updates across multiple windows. Experiments show that while sequential online learning improves forecasts during training, it fails to generalise, whereas the batched formulation consistently enhances out‑of‑sample forecast skill and improves convergence with larger batch sizes.
arXiv:2609.35944v1 Announce Type: new Abstract: Data Assimilation (DA) aims to recover the full state of a dynamical system that is only partially observed. A solution is to use Score-based models to...
arXiv:2608. 05103v1 Announce Type: new Abstract: Data assimilation (DA) uses Bayesian inference to update the state of a numerical forecast model with observed data.
arXiv:2608. 05103v2 Announce Type: replace Abstract: Data assimilation (DA) uses Bayesian inference to update the state of a numerical forecast model with observed data.