arXiv Machine Learning

Amortized Filtering and Smoothing with Conditional Normalizing Flows

arXiv Machine Learning
Sep 24

PR-Smoother: Simulator-Preserving Non-Gaussian Smoothing for Data Assimilation

PR‑Smoother is an amortized smoothing method that preserves the explicit use of a prescribed simulator in both the evidence lower bound and the variational family. It learns only future‑conditioned corrections to the simulator’s rollout, yielding a non‑Gaussian smoothing distribution that can jointly infer state, parameters, and sensor bias from observations alone. The approach recovers the exact smoother in deterministic and linear‑Gaussian limits and has been shown to capture multimodal posteriors in Lorenz‑96 and scale to 16,384‑dimensional Kolmogorov flow.

By Yuta Tarumi
arXiv Machine Learning
Sep 23

FREESIA: Covariance-Aware Posterior Transport for Expressive and Scalable Data Assimilation

The paper introduces FREESIA, a training‑free, covariance‑aware posterior transport method for data assimilation that embeds forecast cross‑covariance into a flow‑based transport to recover unobserved states while preserving non‑Gaussian posterior structure. It combines an observation‑adaptive proposal with posterior correction, providing an asymptotically exact approximation of nonlinear posteriors and a Wasserstein error bound. Experiments on Double‑Well, Lorenz‑96, and Kolmogorov flow demonstrate that FREESIA captures complex posterior structures and achieves up to a 56% reduction in RMSE compared to the best baseline in sparse, nonlinear, non‑injective observation scenarios.

By Shiwei Ni, Yangwen Zhang, Hang Qi, Xiaofei Guan, Lili Ju
arXiv Machine Learning
1d ago

Auto-differentiable data assimilation: Co-learning of states, dynamics, and filtering algorithms

The paper presents a framework called auto‑differentiable filtering that jointly learns the state, dynamics, and filtering algorithm parameters in data assimilation. It uses a loss function that allows gradient‑based optimization from partial, noisy observations via auto‑differentiation. Experiments demonstrate the method on diverse dynamical systems, including aerospace, atmospheric, biological, and geophysical models, and the authors offer guidelines for tailoring the framework to specific observation models, accuracy needs, and computational budgets.

By Melissa Adrian, Daniel Sanz-Alonso, Rebecca Willett