arXiv Machine Learning
Oct 5

AECSF: Adaptive Ensemble Conditional Score Filtering for High-Dimensional Nonlinear Data Assimilation

AECSF: Adaptive Ensemble Conditional Score Filtering for High‑Dimensional Nonlinear Data Assimilation proposes a training‑free adaptive ensemble conditional score filter that constructs an analytically tractable score estimator using the conditional Tweedie identity. The method employs a shared adaptive weighted proposal ensemble to efficiently estimate conditional means for each noisy reverse particle, updating the proposal within the same reverse‑diffusion run. Theoretical analysis characterizes when a fixed weighted proposal yields the exact noisy posterior score and bounds the error between conditional‑mean estimation and reverse‑sampling endpoint error, while numerical experiments show improved posterior sampling and nonlinear filtering accuracy in high‑dimensional problems with limited forecast ensembles.

By Yangwen Zhang, Shiwei Ni, Xiaoping Zhang, Xiaofei Guan, Lili Ju