arXiv Machine Learning
What Information Buys in Active Inference
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 21
Expected Free Energy as Belief-Dependent Utility for rho-POMDPs
arXiv:2607. 16981v1 Announce Type: new Abstract: An agent acting under partial observability must decide when to gather information and which observations are worth their cost.
arXiv Machine Learning
3d ago
The Price of Current-Effect Inference in Nonstationary Multi-Armed Bandits
The paper studies the trade‑off between maximizing rewards and accurately estimating current treatment effects in nonstationary multi‑armed bandits. For $K$‑armed bandits with H"older‑smooth mean rewards, the authors derive minimax bounds on expected dynamic regret under a high‑probability simultaneous coverage requirement for all current effects. They propose the TRACE algorithm, which achieves a regret–precision trade‑off that separates reward‑learning regret from the unavoidable cost of refreshing current‑effect estimates, and extend the method with an empirical‑Bernstein e‑process to adapt exploration while preserving worst‑case guarantees.
arXiv AI
Jul 21
Principled Direction-Free Intrinsic Motivation through Model-Free Epistemic Free-Energy Estimators
arXiv:2607. 16858v1 Announce Type: cross Abstract: Across environments with mixed sources of uncertainty, unsupervised reinforcement learning requires intrinsic motivation that does not precommit to a particular direction of surprise.
arXiv Machine Learning
Aug 19
The concentration game: Bayesian updating, regret, and information
The paper introduces a two-player zero-sum repeated game between a learner and nature that simultaneously captures Bayesian updating and an exact decomposition of exponential-weights regret. The game’s terminal payoff reflects the maximum gain a comparator can achieve given a fixed relative entropy from the prior, while the one-step constraint limits nature’s move by an information budget. The resulting regret splits into three precise components—per-round information loss, an additive retempering drift, and the comparator’s information relative to the prior—providing a unified framework that explains concentration phenomena, large-deviation bounds, and various learning methods such as bandits, posterior sampling, aggregation, and boosting.
arXiv Machine Learning
Jun 2
Online Learning in MDPs with Partially Adversarial Transitions and Losses
arXiv:2602. 09474v2 Announce Type: replace Abstract: We study reinforcement learning in MDPs whose transition function is stochastic at most steps but may behave adversarially at a fixed subset of $\Lambda$ steps per episode.
arXiv Machine Learning
Oct 15
High-Probability Regret Bounds for Feel-Good Thompson Sampling
The paper presents the first high‑probability regret bounds for both cumulative and current‑step variants of Feel‑Good Thompson Sampling (FGTS) in contextual bandits with general realizable function approximation. The bounds rely on decoupling coefficients and prior mass near the true reward function, and in the linear setting they achieve ≠d√T log(2T)+√T log(1/δ) regret with probability at least 1‑δ. This matches the dimension and horizon dependence of upper‑confidence‑bound algorithms up to logarithmic factors, while the high‑probability cost adds only linear dependence on log(1/δ).