arXiv AI
Aug 13
Post-Training with Policy Gradients: Optimality and the Base Model Barrier
arXiv:2603. 06957v2 Announce Type: replace-cross Abstract: We study post-training linear autoregressive models with outcome and process rewards.
Reconciling Universal and Uniform Learning with $Q$-Aggregation
The paper investigates regression with bounded responses, comparing two learning frameworks: model selection aggregation, which requires improper algorithms to achieve minimax excess risk, and universal learning, where empirical risk minimization suffices for exponential learning rates. For finite hypothesis classes, the authors show that the $Q$-aggregation estimator simultaneously attains minimax optimal tails and exponential universal rates, while other common estimators fail to do so. For countably infinite classes, they prove an inherent trade‑off between exponential universal and minimax uniform rates, resolved by combining optimal algorithms from each framework via $Q$-aggregation.
Related stories
arXiv AI
Jul 7
Sign-Separated Asymmetric Finite-Time Error Analysis of Q-Learning
arXiv:2605. 16103v2 Announce Type: replace Abstract: Q-learning is known to suffer from overestimation bias: because the Bellman update maximizes noisy or imperfect action-value estimates, positive errors can be selected and propagated, causing learned values to exceed the true optimal values.
arXiv Machine Learning
Aug 28
Algorithmic Principles For Multiclass Learning Are Hard To Come By: Limits of Regularization and Proper Learning
The paper investigates fundamental limits of algorithmic principles in multiclass learning, specifically proper learning and regularization. It shows that learning cannot always be reduced to proper learning even with an enlarged hypothesis class, that proper learners may need a sublinear number of errors that can be arbitrarily large, and that regularization (SRM or local) is not universally sufficient. The authors also provide a positive theory giving sufficient conditions for SRM learnability and a characterization via integrability of revealed preferences.
arXiv Machine Learning
Aug 18
Last-Iterate Analyses of FTRL with the 1/2-Tsallis Entropy in Stochastic Bandits
arXiv:2510. 22819v3 Announce Type: replace Abstract: The convergence analysis of online learning algorithms is central to machine learning theory, where the last-iterate convergence is particularly important, as it captures the learner's actual decisions and describes the evolution of the learning process over time.
arXiv Machine Learning
Aug 26
An Efficient Minimax-Optimal Algorithm for Adversarial $m$-Set Bandits
arXiv:2608. 12231v2 Announce Type: replace Abstract: We study adversarial combinatorial bandits with $m$-set actions, where at each round the learner selects $m$ out of $d$ items and observes only the aggregate loss of the selected items.
arXiv Machine Learning
Jun 5
Adaptive Learning Rates with Surrogate Probability for Follow-the-Perturbed-Leader
arXiv:2606. 06043v1 Announce Type: cross Abstract: Follow-the-regularized-leader framework has shown effectiveness and flexibility in online learning problems, where the choice of learning rates are known to be crucial.
arXiv Machine Learning
Jun 2
Near-Optimal Pure Machine Unlearning for Smooth Strongly Convex Losses
arXiv:2606. 01527v1 Announce Type: new Abstract: Machine unlearning is motivated by legal and user-facing requirements to remove the influence of individuals' data from trained models, such as the right to be forgotten.
arXiv AI
Jun 16
A Model-Free Universal AI
arXiv:2602. 23242v3 Announce Type: replace Abstract: In general reinforcement learning, all established optimal agents, including AIXI, are model-based, explicitly maintaining and using environment models.
arXiv AI
Aug 7
An Optimal Agnostic PAC Algorithm
arXiv:2608. 06363v1 Announce Type: cross Abstract: Let $H\subseteq\{-1,+1\}^X$ be a class of finite VC dimension $d\ge1$.
arXiv Machine Learning
Jul 21
Theoretical Foundations of $\max$@$k$ Reinforcement Learning
arXiv:2607. 17823v1 Announce Type: new Abstract: Reinforcement Learning is a cornerstone technique for modern large reasoning models.
arXiv Machine Learning
Aug 4
Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning
arXiv:2608. 02034v1 Announce Type: new Abstract: Multi-step returns accelerate reward propagation in off-policy reinforcement learning, but couple the evaluation of each decision to the suboptimal logged actions that follow it, inducing a pessimistic bias that grows with the horizon.
arXiv Machine Learning
Aug 7
Optimal Rates for Learning with Monotone Adversaries
arXiv:2608. 06337v1 Announce Type: cross Abstract: A monotone adversary observes an i.