arXiv AI

The Curvature Shadow: An Apparent Failure of Maximum-Entropy Equilibrium Selection is a Removable Artifact

arXiv:2607. 17543v1 Announce Type: new Abstract: In two-player zero-sum games whose Nash equilibria form a convex set, regularized solvers such as Regularized Nash Dynamics (R-NaD) empirically select the maximum-entropy member: the information projection (I-projection) of a uniform reference onto the Nash set.

arXiv AI
Sep 18

Steering Equilibrium Selection in Regularized Self-Play via the Reference Policy

The paper investigates how a reference policy can be used to steer regularized self‑play toward a specific equilibrium in two‑player zero‑sum games. By anchoring the reference at a target equilibrium and refining the self‑play process, the authors achieve precise convergence to that target with very low exploitability and coordinate error. The study also explores the effects of off‑manifold references, mirror‑step sizing, and boundary saturation on selection accuracy.

By Luis Leal
arXiv Machine Learning
Sep 25

Optimal Recovery Meets Bayesian Learning: Where Worst-Case Bounds Pay Off

The paper shows that Worst‑Case Optimal Recovery (OR) and Bayesian learning solve the same Gaussian‑quadratic‑Hilbert problems, linking the radius of information to a nugget‑optimized Gaussian process posterior variance. It evaluates three Bayesian systems, demonstrating that OR can outperform Bayesian methods in certain calibration and reproducibility metrics, yet split‑conformal and other approaches can beat OR in interval scoring, especially under covariate shift. The authors propose matching the guarantee tool to the data regime and auditing that regime first.

By Gordei Verbii
arXiv Machine Learning
Jun 8

High entropy leads to symmetry-equivariant policies in Dec-POMDPs

arXiv:2511. 22581v5 Announce Type: replace Abstract: We prove that in any Dec-POMDP, sufficiently high entropy regularization ensures that the policy gradient flow with tabular softmax parametrization always converges, for any initialization, to the same joint policy, and that this joint policy is equivariant w.

By Johannes Forkel, Constantin Ruhdorfer, Michael Beukman, Andreas Bulling, Jakob Foerster
arXiv Machine Learning
Aug 10

Multiscale Reward Hedging from Correct Demonstrations

arXiv:2608. 06825v1 Announce Type: new Abstract: Learning from correct demonstrations is harder than supervised learning when many answers are correct: after predicting, the learner sees one valid answer but not whether its own answer was valid, nor any reward.

By Pahan Dewasurendra
arXiv AI
Jun 30

Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models

arXiv:2606. 30627v1 Announce Type: cross Abstract: Conservative offline training is widely advocated as a safe foundation for subsequent online adaptation: if a policy stays close to well-supported behaviour, the argument goes, it is less likely to exploit imperfections in a learned reward model.

By Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary