arXiv Machine Learning

Full-Covariance Smoothing of Bayesian Neural Networks for Online Adaptation

The paper proposes treating a neural network’s layers as time steps in a state‑space model, converting Bayesian training into a smoothing problem. By propagating Gaussian moments forward and applying a Rauch–Tung–Striebel backward pass, weight posteriors are updated in closed form without gradient iterations or replay. The authors extend prior work by introducing a cross‑covariance identity that allows full‑covariance propagation through nonlinear activations, enabling more accurate online adaptation in non‑stationary classification, dynamics learning, and vision‑language‑action policy adaptation.

arXiv Machine Learning

5d ago

Online Learning via Learned Latent Bayesian Tracking

The paper introduces AURA, a meta‑learning framework that learns a low‑dimensional latent state‑space model for the evolution of optimal model parameters under distribution shift. Online adaptation is performed via extended Kalman filtering in this latent space, followed by reconstruction of full model parameters through a learned lifting map, enabling efficient single‑step updates. Experiments on neural wireless receivers and non‑stationary image classification show that AURA improves adaptation speed, accuracy, and computational efficiency compared to existing online learning and Bayesian filtering baselines.

By Guy Gerson, Tomer Raviv, Nir Shlezinger, Tirza Routtenberg, Osvaldo Simeone

diffusion computer-vision

arXiv AI

Jun 2

Computation-Aware Kalman Filtering with Model Selection for Neural Dynamics

arXiv:2606. 01468v1 Announce Type: cross Abstract: Due to their explicit priors and ability to model uncertainty, Bayesian methods have played a major role in dynamical latent variable modeling of single-cell neural recordings.

By JR Huml, Jonathan Wenger, John P. Cunningham

arXiv AI

Sep 10

Kalman Delta Networks: Uncertainty-aware Associative Memory

Kalman Delta Networks (KDNs) extend linear attention models by treating associative memory as a linear–Gaussian state‑space system, enabling the Kalman filter to optimally estimate both memory state and its uncertainty. Two GPU‑friendly approximations—Diagonal KDN and Isotropic KDN—use mean‑field variational inference or a single scalar uncertainty per head, respectively, to maintain tractable uncertainty recurrences during linear‑attention scans. Experiments on 750 M and 1.3 B‑parameter models show that KDN variants consistently lower perplexity and raise downstream accuracy compared to existing linear‑attention baselines.

By Ngoc Bui, Tinglin Huang, Rex Ying

llms rag benchmarks

arXiv Machine Learning

Sep 21

Amortized Filtering and Smoothing with Conditional Normalizing Flows

arXiv:2604.07169v3 Announce Type: replace-cross Abstract: Bayesian filtering and smoothing are central to data assimilation in nonlinear dynamical systems. Recent advances in deep generative models p...

By Tiangang Cui, Xiaodong Feng, Chenlong Pei, Xiaoliang Wan, Tao Zhou

diffusion

arXiv AI

Jul 7

Explainable Bayesian deep learning through input-skip Latent Binary Bayesian Neural Networks

arXiv:2503. 10496v2 Announce Type: replace-cross Abstract: Modeling natural phenomena with artificial neural networks (ANNs) often provides highly accurate predictions.

By Eirik H{\o}yheim, Lars Skaaret-Lund, Solve S{\ae}b{\o}, Aliaksandr Hubin

benchmarks

arXiv Machine Learning

Jun 2

Fixed-Mean Gaussian Processes for Post-hoc Bayesian Deep Learning

arXiv:2412. 04177v2 Announce Type: replace Abstract: Recently, there has been an increasing interest in performing post-hoc uncertainty estimation about the predictions of pre-trained deep neural networks (DNNs).

By Luis A. Ortega, Sim\'on Rodr\'iguez-Santana, Daniel Hern\'andez-Lobato

benchmarks

Hugging Face Trending Papers

Jul 8

An optimal control approach for neural network architecture adaptation with a posteriori error estimation

This work presents a novel approach for adapting neural network architecture along the depth based on a posteriori error estimation. By formulating neural network training as a continuous-time optimal control problem, we derive rigorous error estimates that quantify how approximation error distributes across network layers.

arXiv Machine Learning

Jun 26

Learning Probabilistic Filters with Strictly Proper Scoring Rules

arXiv:2606. 26497v1 Announce Type: new Abstract: Bayesian filtering of partially and noisily observed dynamical systems seeks to infer the evolving conditional distribution of the state of a dynamical system, given observations, in an online fashion.

By Eviatar Bach, Ricardo Baptista, Jochen Br\"ocker, Bohan Chen, Andrew Stuart

llms multimodal safety

arXiv AI

Sep 4

Subspace Inference Enables Efficient Active Reward Learning from Preferences

The paper introduces PreferenceEKF, a sample‑efficient method for active reward learning from human preferences. By framing preference learning as a sequential Bayesian filtering problem, it tracks reward model uncertainty using an extended Kalman filter in a low‑dimensional subspace, avoiding costly posterior inference over the full neural network. Experiments on D4RL and V‑D4RL benchmarks show improved sample efficiency, runtime, scalability, and calibration, with reward models that support competitive offline reinforcement learning policies.

By Yutai Zhou, Erdem B{\i}y{\i}k

reinforcement-learning benchmarks

arXiv Machine Learning

Jul 9

An optimal control approach for neural network architecture adaptation with a posteriori error estimation

arXiv:2607. 07637v1 Announce Type: new Abstract: This work presents a novel approach for adapting neural network architecture along the depth based on a posteriori error estimation.

By C G Krishnanunni, Thomas Scott, Tan Bui-Thanh

arXiv Machine Learning

Jun 15

Uncertainty Estimation and Generalization Bounds for Modern Deep Learning

arXiv:2606. 13818v1 Announce Type: new Abstract: This thesis investigates how Bayesian principles can deepen our understanding of modern deep learning systems.

By Luis A. Ortega

arXiv AI

Jun 6

Retry Policy Gradients in Continuous Action Spaces

arXiv:2606. 05888v1 Announce Type: new Abstract: Retry-based objectives such as pass@K and max@K optimize the best return obtained from multiple sampled trajectories, and recent work has shown that they can promote exploration without explicit exploration bonuses.

By Soichiro Nishimori, Paavo Parmas

Related stories