The paper critiques the use of pass@$k$ as a sole metric for evaluating reinforcement‑learning post‑training on language models, showing that it ignores diversity in model outputs. Experiments on Qwen2.5‑1.5B‑Instruct trained with Group Relative Policy Optimization (GRPO) and rejection‑sampling fine‑tuning (RFT) reveal that these methods shift token‑level, answer‑level, and unique‑answer diversity in opposite directions, yet pass@$k$ fails to capture these differences. Even when restricting to verifier‑correct completions, diversity gaps persist, and pass@$k$ does not consistently identify the better performing arm on hard problem subsets, indicating that the metric can mask true capability changes.
By Subham Rath, Raj Dandekar, Rajat Dandekar, Sreedath Panat
arXiv:2607.09709v2 Announce Type: replace
Abstract: Post-training a code generator against a learned judge can optimize proxy features that raise the score without improving the artifact. We study th...
By Chenyu Zhou, Qiliang Jiang, Shuning Wu, Xu Zhou
arXiv:2607. 09709v1 Announce Type: new Abstract: Post-training a code generator against a learned judge can optimize proxy features that raise the score without improving the artifact.
By Chenyu Zhou, Qiliang Jiang, Shuning Wu, Xu Zhou
arXiv:2609.06107v1 Announce Type: new
Abstract: Data policies for reinforcement learning with verifiable rewards (RLVR) determine which rollouts are used, how strongly they are weighted, and which do...
By Hao Liang, Mingrui Chen, Hengyi Feng, Meiyi Qiang, Wentao Zhang
arXiv:2605. 30612v2 Announce Type: replace-cross Abstract: Continuous control policies trained with off-policy reinforcement learning frequently exhibit high-frequency action jitter, impractical for direct deployment on physical actuators.
By Faiq Shamass
The paper investigates how advantage scale calibration can become imbalanced in group-relative optimization when rewards have low variance. It distinguishes two problematic low‑variance scenarios—sub‑resolution jitter and small but credible cardinal gaps—and introduces a three‑way calibration interface that links within‑group scale to reward‑branch strength, prompt‑level batch weight, and KL calibration. Building on this interface, the authors propose the Reward‑Resolution Protocol and MaxNorm‑AC to filter out sub‑resolution gaps and provide bounded recovery for credible gaps, achieving better performance than existing robust‑scale baselines across dense, MoE, math, and code reasoning tasks.
By Fei Ding
The paper introduces CARE, a method for certifying accelerated inference in vision‑language‑action models. CARE uses paired rollouts on a calibration set to guarantee that the risk of acceleration‑induced failure stays below a user‑specified budget, selecting the fastest certified accelerator or falling back to the reference policy. Experiments on LIBERO suites show that CARE achieves 9.0–10.8× speedups while preserving at least 85.8% of reference‑solved episodes with 95% confidence, and it outperforms non‑guaranteed selectors in both budget adherence and rollout efficiency.
By Rui Liu, Tong Zheng, Jindong Gu, Zhipeng Wang
arXiv:2607. 21273v2 Announce Type: replace Abstract: Dense per-step supervision is the standard remedy for sparse-reward long-horizon LLM agents: reward the policy for predicting its next observation, which looks provably safe under potential-based shaping.
By Yu Wang
arXiv:2610.00385v1 Announce Type: cross
Abstract: Replay selectors often rank cached trajectories by format feedback, confidence, freshness, or response length, although cache-level correctness and d...
By Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Tianshu Fu, Daren Zha, Jun Xiao
While vision-language-action (VLA) models have advanced rapidly, running them at every control step remains expensive. Prior work accelerates VLA inference using techniques like action chunking and vi...
The paper investigates selective on‑policy distillation, where a student model is trained only on token positions chosen by a selector. It demonstrates that the commonly used shared learning rate is not neutral: performance varies significantly with the learning rate for different selectors, leading to inconsistent comparisons. The authors attribute this selector‑rate entanglement to the selection process itself and recommend reporting the full arm‑by‑rate matrix for fair evaluation.
By Chencheng Zhu
SimplexUQ introduces the first benchmark and reproducible protocol for evaluating how conformal prediction wrappers allocate coverage across simplex‑valued predictions. The framework, called SimplexTasks‑12, combines six synthetic regimes and six real tasks (e.g., class probabilities, topic mixtures, spectral abundances) to compare existing wrappers on metrics such as marginal coverage, worst‑stratum coverage, max disparity, and computational cost. Empirical results show that no single wrapper consistently dominates, with Mondrian and BatchMVP performing best in different settings, and that removing predictor bias only partially mitigates disparity.
By Liang You, Hengyu Shi, Dongwen Ou