arXiv Machine Learning By Sumit Vashishtha

Bayesian Neural Networks via Dirichlet Process Priors on the Data-Generating Distribution for Randomized Value Functions in Deep Reinforcement Learning

Read the original on arXiv Machine Learning →

The paper introduces a new class of Bayesian neural networks that place Dirichlet Process priors on the data-generating distribution instead of on network weights, aiming to better capture epistemic uncertainty. Using this framework, the authors develop DP-DQN, a reinforcement learning algorithm that maintains posterior distributions over value functions and performs efficient Thompson sampling. DP-DQN combines data-driven posterior reweighting via the Bayesian Bootstrap with generative prior hypotheses from the DP base measure, enabling deep exploration with a single neural network without requiring multi-network ensembles.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Sep 4

Subspace Inference Enables Efficient Active Reward Learning from Preferences

The paper introduces PreferenceEKF, a sample‑efficient method for active reward learning from human preferences. By framing preference learning as a sequential Bayesian filtering problem, it tracks reward model uncertainty using an extended Kalman filter in a low‑dimensional subspace, avoiding costly posterior inference over the full neural network. Experiments on D4RL and V‑D4RL benchmarks show improved sample efficiency, runtime, scalability, and calibration, with reward models that support competitive offline reinforcement learning policies.

By Yutai Zhou, Erdem B{\i}y{\i}k