arXiv AI
Jun 10
Baseline-Free Policy Optimization for Neural Combinatorial Optimization
arXiv:2606. 10321v1 Announce Type: cross Abstract: Neural combinatorial optimization (NCO) trains autoregressive policies to solve routing problems.
Stabilized Best-of-$K$ Training for Neural Combinatorial Optimization
arXiv:2608. 00296v1 Announce Type: new Abstract: Leader Reward modifies POMO training to emphasize the best trajectory produced by repeated inference.
Related stories
arXiv Machine Learning
Jul 14
Rank-Conditioned Sample Reuse for the Plackett--Luce Best-of-$K$ Objective
arXiv:2607. 11146v1 Announce Type: new Abstract: We study the coupled objective J_K^WOR = E_{S ~ PL-WOR_K}[max_{i in S} R_i]: the expected maximum reward of a size-K Plackett-Luce draw without replacement, the law of Gumbel-Top-K / Stochastic Beam Search decoding.
arXiv AI
Jul 29
Less Data, Better Alignment: Data-Centric Multi-Evaluator Agreement for Preference Optimization
arXiv:2607. 25136v1 Announce Type: new Abstract: Research on preference optimization often varies the training objective while holding the data fixed.
arXiv Machine Learning
Aug 11
CODS: Iterative Bellman-Residual Data Selection for Reusable Offline Reinforcement Learning
arXiv:2608. 07719v1 Announce Type: new Abstract: Offline reinforcement learning repeatedly trains policies from a fixed transition pool, making redundant data costly across seeds and hyperparameters, while naive subsampling can remove rare transitions needed for long-horizon credit assignment.
arXiv Machine Learning
Aug 20
Many Optimizers But Only One Training Path: Repeated Resampling for Adaptive Optimizer Selection
The paper introduces Repeated Optimizer Resampling (ROR), a method that treats optimizer choice as a hyperparameter and searches for the best optimizer during a single training run. ROR periodically scouts each candidate optimizer for a short number of epochs, then continues training with the best scout, allowing the optimizer to change over time. Experiments on MNIST, Fashion‑MNIST, and motor insurance claim‑count models show that one‑epoch ROR uses only 24–35% of the training effort required to exhaustively evaluate all optimizers while achieving comparable performance.
arXiv Machine Learning
Jun 5
On Advantage Estimates for Max@K Policy Gradients
arXiv:2606. 06080v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards is widely used for post-training reasoning models, but sparse outcome rewards make exploration difficult.
arXiv AI
Jun 29
When Is an LLM Worth It for Hyperparameter Optimization? A Budget-Matched Study on Tabular Data Finds the Warm-Start Is a Default Configuration, Not the Model
arXiv:2606. 21641v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have been proposed as hyperparameter-optimization (HPO) advisors that "warm-start" search from prior knowledge, proposing strong configurations in very few evaluations.
arXiv Machine Learning
Jul 31
LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts
arXiv:2607. 27787v1 Announce Type: new Abstract: Reinforcement learning from verifiable rewards (RLVR) for mathematical reasoning suffers from a structural blind spot: on "cliff" prompts-those on which every sampled rollout in a group fails-the group-normalized advantage is identically zero, so GRPO produces no gradient on precisely the prompts at the frontier of the model's capability.
arXiv AI
Jun 2
Cast a Wider Net: Coordinated Pass@K Policy Optimization for Code Reasoning
arXiv:2605. 27000v2 Announce Type: replace-cross Abstract: Repeated sampling with a verifier is the standard way to allocate test-time compute for code generation, with pass@$K$ as the canonical metric.
arXiv Machine Learning
Jun 4
ALMAB-DC: Active Learning, Multi-Armed Bandits, and Distributed Computing for Sequential Experimental Design and Black-Box Optimization
arXiv:2603. 21180v4 Announce Type: replace Abstract: Sequential experimental design under expensive, gradient-free objectives is a central challenge in computational statistics: evaluation budgets are tightly constrained and information must be extracted efficiently from each observation.
arXiv AI
Jun 9
Adaptive Loss Balancing for Noise-Robust GRPO in Generative Recommendation
arXiv:2606. 08480v1 Announce Type: cross Abstract: Reinforcement learning (RL) presents a promising avenue for enhancing generative recommendation beyond supervised imitation, leveraging reward signals to guide policy improvement.
arXiv Machine Learning
Jul 21
Rethinking Efficiency in Neural Combinatorial Optimization: Batched Preference Optimization with Mamba
arXiv:2602. 20730v3 Announce Type: replace Abstract: We study efficiency as a first-class objective in Neural Combinatorial Optimization (NCO) and present ECO, an efficient learning framework that combines batched preference optimization with a Mamba backbone.