The paper investigates how a reference policy can be used to steer regularized self‑play toward a specific equilibrium in two‑player zero‑sum games. By anchoring the reference at a target equilibrium and refining the self‑play process, the authors achieve precise convergence to that target with very low exploitability and coordinate error. The study also explores the effects of off‑manifold references, mirror‑step sizing, and boundary saturation on selection accuracy.
arXiv:2606. 28308v1 Announce Type: cross Abstract: Many two-player zero-sum games admit not a unique Nash equilibrium but a convex set of them: a polytope of profiles that all share the minimax value V* yet prescribe different behaviour.
arXiv:2607. 17543v1 Announce Type: new Abstract: In two-player zero-sum games whose Nash equilibria form a convex set, regularized solvers such as Regularized Nash Dynamics (R-NaD) empirically select the maximum-entropy member: the information projection (I-projection) of a uniform reference onto the Nash set.
The paper introduces Anchored Neighborhood Optimization (ANO), a new policy‑optimization method that directly designs a smooth, bounded gain field for the probability‑ratio surrogate objective. ANO anchors the identity map at a ratio of one, peaks at a specified trust‑region boundary, and limits the influence of extreme off‑policy samples while providing a bounded, redescending pull on outliers. Empirical results show ANO consistently outperforms existing methods on Atari and MuJoCo benchmarks, and it remains robust under aggressive learning‑rate settings.
arXiv:2503.00030v3 Announce Type: replace-cross Abstract: Self-play-based policy optimization has emerged as an effective approach for fine-tuning large language models (LLMs), formulating preference...
The paper argues that in multi‑turn agentic reinforcement learning, credit assignment should be viewed as a coverage problem rather than a targeting problem. It introduces verifier information density (V_d) as a structural metric, showing that terminal‑state verifiers operate in a low‑V_d regime where targeting fails. Experiments on tau^2‑bench, BFCL, and ToolACE‑2‑8B demonstrate that uniformly distributing reward across all turns outperforms sparse, targeted rewards, and that full chain coverage is necessary for optimal performance.
arXiv:2606. 12502v1 Announce Type: cross Abstract: We propose that value -- the quantity goal-directed agents create, destroy, and exchange -- is a lawful structural quantity in the same category as information.
arXiv:2608. 09586v1 Announce Type: new Abstract: The Independent Chip Model (ICM) converts tournament chips into reference prize equity, and policies are routinely constructed against those values.
arXiv:2607. 11920v1 Announce Type: cross Abstract: Evaluating decisions made under uncertainty is hard when labeled outcomes are scarce, costly, or confounded with luck.
arXiv:2608.27757v1 Announce Type: new Abstract: Searchless chess networks reach human master strength from a single forward pass by imitating a stronger teacher: the strongest, Leela Chess Zero's (Lc...
The paper studies a stationary decentralized Markov game where a focal agent experiences drifting rewards and dynamics due to learning peers, framing this as an agent‑centric continual reinforcement‑learning problem. It introduces the concept of an invariant core—maximal abstract patterns common to many successful trajectories—and proves a worst‑case conditioning theorem linking trajectory‑law drift to success coverage. The authors provide theoretical guarantees for survival horizon, first‑exit law, and regret, and validate their predictions with solvable models and empirical studies in continual control, cue‑MNIST, and Level‑Based Foraging.
arXiv:2607. 05904v1 Announce Type: new Abstract: Training a language model against its own reference-free judgments (the premise of self-rewarding, self-play, and LLM-as-a-judge pipelines) assumes a model's verdict on a shown answer tracks correctness.
arXiv:2608. 08103v1 Announce Type: new Abstract: Smooth acyclicity constraints answer whether a weighted support is a DAG, whereas structure learning asks which support change should be made.