arXiv Machine Learning

Bayesian Neural Networks via Dirichlet Process Priors on the Data-Generating Distribution for Randomized Value Functions in Deep Reinforcement Learning

The paper introduces a new class of Bayesian neural networks that place Dirichlet Process priors on the data-generating distribution instead of on network weights, aiming to better capture epistemic uncertainty. Using this framework, the authors develop DP-DQN, a reinforcement learning algorithm that maintains posterior distributions over value functions and performs efficient Thompson sampling. DP-DQN combines data-driven posterior reweighting via the Bayesian Bootstrap with generative prior hypotheses from the DP base measure, enabling deep exploration with a single neural network without requiring multi-network ensembles.

arXiv AI

Sep 4

Subspace Inference Enables Efficient Active Reward Learning from Preferences

The paper introduces PreferenceEKF, a sample‑efficient method for active reward learning from human preferences. By framing preference learning as a sequential Bayesian filtering problem, it tracks reward model uncertainty using an extended Kalman filter in a low‑dimensional subspace, avoiding costly posterior inference over the full neural network. Experiments on D4RL and V‑D4RL benchmarks show improved sample efficiency, runtime, scalability, and calibration, with reward models that support competitive offline reinforcement learning policies.

By Yutai Zhou, Erdem B{\i}y{\i}k

reinforcement-learning benchmarks

arXiv AI

Jun 2

Possibilistic Predictive Uncertainty for Deep Learning

arXiv:2605. 00600v2 Announce Type: replace-cross Abstract: Deep neural networks achieve impressive results across diverse applications, yet their overconfidence on unseen inputs necessitates reliable epistemic uncertainty modeling.

By Yao Ni, Jeremie Houssineau, Yew Soon Ong, Piotr Koniusz

benchmarks

arXiv Machine Learning

Jul 30

Collaborative Weighting with Pessimistic Critic for Mitigating Overestimation in Off-Policy Reinforcement Learning

arXiv:2607. 26509v1 Announce Type: new Abstract: Deep off-policy reinforcement learning algorithms for continuous control typically rely on neural value function approximation to guide policy improvement.

By Gong Gao, Xiao Lai, Ziqi Xie, Guojie Chen, Xianhui Liu, Weidong Zhao

reinforcement-learning safety

arXiv AI

Jun 2

Regularized Offline Policy Optimization with Posterior Hybrid Bayesian Belief

arXiv:2606. 00680v1 Announce Type: new Abstract: Offline reinforcement learning (RL) aims to optimize policies from pre-collected datasets.

By Hongqiang Lin, Pengfei Wang, Nenggan Zheng

reinforcement-learning benchmarks

arXiv Machine Learning

Jul 21

Information-Based Exploration via Random Features for Reinforcement Learning

arXiv:2607. 17981v1 Announce Type: new Abstract: Representation learning has enabled classical exploration strategies to be extended to deep Reinforcement Learning (RL), but often makes algorithms more complex and theoretical guarantees harder to establish.

By Waris Radji, Odalric-Ambrym Maillard

reinforcement-learning

arXiv Machine Learning

Sep 1

Amortizing intractable inference in large language models

arXiv:2310.04363v3 Announce Type: replace Abstract: Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This limits t...

By Edward J. Hu, Moksh Jain, Eric Elmoznino, Younesse Kaddar, Guillaume Lajoie, Yoshua Bengio, Esmeralda S. Whitammer

llms agents reinforcement-learning fine-tuning

arXiv AI

Jul 3

Full Bayesian Reinforcement Learning via LF-IBIS

arXiv:2607. 01741v1 Announce Type: cross Abstract: Reinforcement Learning (RL) is a sequential decision-making framework in which an agent learns optimal policies through interaction with an environment by maximizing cumulative rewards.

By Stefano Masini, Cecilia Viscardi, Michela Baccini

agents reinforcement-learning

arXiv AI

Sep 10

BaNEL: Exploration Posteriors for Generative Modeling Using Only Negative Rewards

arXiv:2510.09596v2 Announce Type: replace-cross Abstract: Today's generative models thrive with large amounts of supervised data and informative reward functions characterizing the quality of the gen...

By Sangyun Lee, Brandon Amos, Giulia Fanti

arXiv AI

Jun 2

Computation-Aware Kalman Filtering with Model Selection for Neural Dynamics

arXiv:2606. 01468v1 Announce Type: cross Abstract: Due to their explicit priors and ability to model uncertainty, Bayesian methods have played a major role in dynamical latent variable modeling of single-cell neural recordings.

By JR Huml, Jonathan Wenger, John P. Cunningham

arXiv Machine Learning

Jul 31

On-Policy and Off-Policy Learning for Large Action Spaces

arXiv:2607. 28408v1 Announce Type: new Abstract: This thesis studies policy learning in interactive systems where an agent observes a context, selects an action from a very large set, and receives partial feedback.

By Imad Aouali

agents diffusion safety

arXiv AI

Jun 15

Deep Dense Exploration for LLM Reinforcement Learning via Pivot-Driven Resampling

arXiv:2602. 14169v2 Announce Type: replace-cross Abstract: Effective exploration is a key challenge in reinforcement learning for large language models: discovering high-quality trajectories within a limited sampling budget from the vast natural language sequence space.

By Yiran Guo, Zhongjian Qiao, Yingqi Xie, Jie Liu, Dan Ye, Ruiqing Zhang, Shuang Qiu, Lijie Xu

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Oct 8

Sparsifying Stochasticity, Not Capacity: Partial Stochasticity via Deep Weight Factorization of Prior Scales

arXiv:2610.09886v1 Announce Type: cross Abstract: Bayesian neural networks need not be fully stochastic to be universal conditional density approximators, but it remains open which parameters should...

By Marius P Linhard, Maurizio Filippone

benchmarks

Related stories