arXiv Machine Learning

Unscented KalmanNet: a hybrid deep learning filter with calibrated posterior covariance for nonlinear state estimation

arXiv:2608. 04201v1 Announce Type: new Abstract: State estimation for nonlinear dynamical systems is commonly performed with the Unscented Kalman filter (UKF), which propagates the state moments through deterministic sigma points and reports a posterior covariance at every step.

arXiv Machine Learning
Sep 24

Full-Covariance Smoothing of Bayesian Neural Networks for Online Adaptation

The paper proposes treating a neural network’s layers as time steps in a state‑space model, converting Bayesian training into a smoothing problem. By propagating Gaussian moments forward and applying a Rauch–Tung–Striebel backward pass, weight posteriors are updated in closed form without gradient iterations or replay. The authors extend prior work by introducing a cross‑covariance identity that allows full‑covariance propagation through nonlinear activations, enabling more accurate online adaptation in non‑stationary classification, dynamics learning, and vision‑language‑action policy adaptation.

By Oren Wright, Haoming Jing, Qiaoan Shen, Koichiro Niinuma, Yorie Nakahira, Jos\'e M. F. Moura
arXiv AI
Jun 30

Learning to Distributedly Estimate under Partially Known Dynamics: A Covariance-Agnostic Neural Kalman Consensus Filter

arXiv:2606. 28441v1 Announce Type: cross Abstract: Online latent state estimation constitutes a fundamental challenge within the artificial intelligence field, serving as a foundational tool for diverse applications, including sequential decision making, anomaly and change-point detection.

By George Stamatelis, Kyriakos Stylianopoulos, George C. Alexandropoulos
arXiv AI
Sep 10

Kalman Delta Networks: Uncertainty-aware Associative Memory

Kalman Delta Networks (KDNs) extend linear attention models by treating associative memory as a linear–Gaussian state‑space system, enabling the Kalman filter to optimally estimate both memory state and its uncertainty. Two GPU‑friendly approximations—Diagonal KDN and Isotropic KDN—use mean‑field variational inference or a single scalar uncertainty per head, respectively, to maintain tractable uncertainty recurrences during linear‑attention scans. Experiments on 750 M and 1.3 B‑parameter models show that KDN variants consistently lower perplexity and raise downstream accuracy compared to existing linear‑attention baselines.

By Ngoc Bui, Tinglin Huang, Rex Ying
arXiv Statistics ML
Sep 3

The Ensemble Kalman Inversion Race

The paper compares different Ensemble Kalman methods for calibrating climate model parameters by minimizing the misfit between modeled and observed climate statistics. It conducts systematic numerical experiments on Lorenz-type models, including neural network parameterizations, to evaluate computational efficiency and accuracy of each method. The study examines how prior information and dimensionality affect the cost of these methods.

By Rebecca Gjini, Matthias Morzfeld, Oliver R. A. Dunbar, Tapio Schneider