The paper introduces ModeBench, a benchmark for multi-solution tasks that evaluates both correctness and the diversity of solutions found by a language model. It shows that reinforcement learning with verifiable rewards (RLVR) tends to collapse solution modes, concentrating probability on fewer correct answers even while accuracy remains high. To counter this, the authors propose Re:Max, a replay-based method that stores one verified example per discovered mode and trains uniformly on these stored modes, improving both success rates and solution diversity across various model scales and RL objectives.
By Liv G. d'Aliberti, Marwa Abdulhai, Sofiia Druchyna, Peter Henderson, Manoel Horta Ribeiro
The paper establishes a general minimax regret lower bound of Ω(√{Tγ_T/ log T}) for kernel bandits with non‑constant continuous kernels on compact domains, matching existing upper bounds up to logarithmic factors. It shows that this log factor is unavoidable in general but can be removed under specific conditions. Consequently, the optimal scaling is exactly Θ(√{Tγ_T}) for several common kernels, including Matérn‑ν (ν∈(0,2)), γ‑exponential (γ∈(0,2)), and certain piecewise‑polynomial kernels.
By Chenkai Ma, Jonathan Scarlett
The paper presents Conversational Voice Aesthetic Model (CVAM), a speech large language model that describes voice aesthetics in conversational contexts. CVAM predicts nine categorical attributes such as gender, pitch, pacing, emotion, and delivery, and is trained on human-annotated data from the CANDOR corpus. Experiments show that CVAM aligns better with human listeners than Gemini 3.1 Pro and other open‑source speech LLMs, demonstrating the value of grounding voice aesthetics in human perception.
By Xilin Jiang, Shun Zhang, Tejas Jayashankar, Yinghao Aaron Li, Osama Hanna
FAITH is a model‑free reinforcement learning framework that separates safety from task performance by using a learned safety filter. The filter approximates the optimal state‑action safety value with a feedforward network, allowing the task policy to optimize return through filtered dynamics without a competing safety term. Experiments on a double integrator, Safety Gym, a 29‑DoF humanoid, and a real Unitree G1 show that FAITH achieves high returns while maintaining near‑perfect safety and handling infeasible starts gracefully.
By Songyuan Zhang, Baljeet Singh, Sarthak Ranjeet Kaingade, Chuchu Fan, Bryan Trinh
arXiv:2610. 12153v1 Announce Type: new Abstract: We study episodic constrained Markov decision processes with adversarial losses under stochastic hard constraints.
By Qian Zuo, Francesco Emanuele Stradi
StoreBench is a live‑commerce environment where an autonomous agent runs a mid‑size online apparel store on a production‑grade backend, facing continuous customer orders, supplier price changes, and market shocks. The agent uses the same 29 merchant tools as a human operator, with a windowed operation budget that ties simulated time to actions, ensuring that model latency does not affect the simulation. Seven large language models were evaluated across 11 scenarios of 30–45 days and a full simulated year, but none matched the scripted smart‑triage policy, and human experts outperformed all models.
By Daksh Raghuvanshi, Ved Vedere, Yifan Wang
The paper introduces CARing, a framework that improves next‑visit diagnosis prediction by representing diagnoses with compositional Semantic IDs (SIDs) and optimizing multi‑label coverage through reinforcement learning. CARing encodes ontology‑enriched disease semantics into compact SIDs, aligns them with natural language and EHR contexts, and employs a coverage reward to encourage diverse diagnosis predictions. On MIMIC‑III and MIMIC‑IV datasets, CARing outperforms all EHR‑trained baselines in weighted F1 and achieves the highest top‑k recall, including R@30 scores above 46% in reasoning mode.
By Kaisong Zhang, Haotian Fang, Junmeng Zhou, Hang Lv, Yulan Pan, Yanchao Tan
The paper compares classical motion planners with learning-based methods for real-time navigation in environments with dynamic hazards, using a unified benchmark that includes identical settings and metrics. In deterministic scenarios, classical planners achieve near-perfect success and high-quality paths, though they can incur significant planning time. When obstacle dynamics are stochastic, classical planners become highly budget-sensitive, whereas PPO-trained policies consistently deliver better latency, success rates, and path quality.
By Eran Iceland, Alexander Tuisov, Oren Gal, Ariel Barel, Alfred M. Bruckstein
The paper investigates why on‑policy distillation (OPD) can fail, observing that larger‑scale teachers often hit early loss plateaus while self‑RL teachers achieve much greater loss reductions. By modeling OPD as a continuous‑time dynamical system and analyzing training logs, the authors link plateaus to a rapid decline in a gradient‑based learning‑signal proxy, even though loss remains high. They also prove a local recovery guarantee for teachers close to the initial student and note that minimal parameter changes and high representation similarity suggest limited adaptation may cause learning‑signal collapse.
By Lei Zhao, Qichao Zhao, Bowen Zuo, Qishi Zhan
The paper introduces Transport REINFORCE, a model‑free policy‑gradient method for discrete‑time mean‑field control that captures both the dynamics and population‑distribution effects of a policy. It perturbs the population distribution—directly on the simplex for finite state spaces and via a Gaussian‑mixture transport map for continuous spaces—to estimate the missing mean‑field contribution. The authors prove consistency of the perturbed objective and gradient, provide bias and mean‑square error bounds, and demonstrate improved performance over standard REINFORCE on several benchmarks.
By Adonis Jamal, Samy Mekkaoui, Yadh Hafsi, Huy\^en Pham
CAPABLE is a capability‑aware policy adaptation framework that enables frozen vision‑language‑action (VLA) policies to recover from joint faults without task‑specific retraining or fault labels. It self‑infers each joint’s actual motion contribution using a temporal encoder, Jacobian grounding, and cross‑joint attention, then conditions a residual reinforcement learning policy to apply bounded corrections to the VLA arm action. Experiments on 28 LIBERO tasks show CAPABLE improves success rates from 24.8% to 59.3% on an actuator excluded from fault training, outperforming a global‑history baseline by 17.4 points while maintaining healthy performance and demonstrating transfer to unseen fault families and a physical Franka Panda robot.
By Mohammad Khoshnazar, Mohammad Dehghani Tezerjani, Deyuan Qu, Zhiyuan Gao, Yanxiang Zhan, Jeroen Schafer, Andrew Melnik, Qing Yang, Michael Beetz
Delphos is a deep reinforcement learning framework designed to aid the specification of discrete choice models. It treats model specification as a sequential decision-making task, where an agent selects modeling actions—such as adding constants, choosing taste parameters, applying transformations, or including interactions—to generate candidate models. Each candidate is estimated and evaluated using a reward function that reflects statistical fit and behavioral expectations, and a Deep Q-Network learns which sequences of decisions lead to high-quality models, as demonstrated on simulated and empirical datasets.
By Gabriel Nova, Stephane Hess, Sander van Cranenburgh
The paper investigates whether small large language model agents can operate within explicit wall‑clock time budgets while using the allotted time productively. Experiments with Qwen3.6‑27B on MLE‑Bench Lite and Qwen3‑4B on Zork I show that agents fail to translate budget statements into controlled time use, largely due to lack of timing feedback and poor anticipation of action durations. Interventions such as harness‑based timing exposure, enforcement hooks, and reinforcement learning with budget‑aware rewards improve budget adherence but do not lead to better task performance, revealing a persistent gap between respecting time limits and effectively allocating extra time.
By Aaron Wang, Neelabh Madan, Vlad Sobal, Matthew Trager, Michael Kleinman, Elman Mansimov, Wei Xia, Stefano Soatto
The paper introduces GenUI‑Harness, a multi‑agent system that pairs a Tool Agent for data retrieval with a GUI Coder Agent that generates structured interfaces to aid human‑agent interaction. It tackles training challenges by using Dynamic UX for lightweight reward collection and Reward Auditor to monitor reward quality. The authors present UI‑TAU Bench, a benchmark of 1,300 database‑backed tasks, and show that GenUI‑Harness improves Pass@3 scores and reduces dialogue rounds compared to existing models.
By Xiaolong Li, Xiaohan Xu, Jinyang Li, Xinnuo Xu, Ge Qu, Nan Huo, Jack Williams, Reynold Cheng
RL-ARC is a calibration‑aware training framework that jointly uses reasoning confidence and answer confidence to improve the calibration of large reasoning models. It applies reasoning‑guided regularization for correct predictions and an overconfidence penalty for incorrect ones. Experiments on in‑distribution and out‑of‑distribution data show that RL‑ARC enhances calibration while preserving reasoning performance.
By Gukhyeon Lee, SangKeun Lee
The paper introduces a three‑dimensional framework for comparing AI methods that solve sequential decision‑making puzzles. It maps each approach to a Markov decision process, evaluates autonomy via human ranking, and measures skill against computational cost. Applying this to the Tower of Hanoi, the authors compare graph‑based, reinforcement learning, and large language model techniques, finding that LLMs shift complexity to inference time, incurring higher memory and runtime costs.
By Sadig Gojayev, Carolina Fortuna
The paper introduces DRL-DCO, a Deep Reinforcement Learning‑governed dynamic orchestration of Differential Evolution and CMA‑ES for high‑dimensional structural model updating. A DDPG actor‑critic agent uses a progression‑aware state and diversity‑informed reward to continuously reallocate resources between exploration and exploitation, while controlling population size, elite preservation, and restarts. The trained actor can operate in a supervision‑free inference mode, enabling fast deployment with maintained performance, and demonstrates superior convergence accuracy and robustness on benchmark optimization and structural health monitoring problems.
By Lechen Li (State Key Laboratory of Internet of Things for Smart City, University of Macau, Macau 519000, China, College of Water Conservancy and Hydropower Engineering, Hohai University, Nanjing 210098, China), Rongye Shi (School of Artificial Intelligence, Beihang University, Beijing 100191, China), Wanhuan Zhou (State Key Laboratory of Internet of Things for Smart City, University of Macau, Macau 519000, China)
LiteGUI is a new framework for training lightweight GUI agents that tackles the long‑horizon, multi‑path nature of GUI interactions. It introduces a two‑stage post‑training pipeline: Guided On‑Policy Distillation, which uses human‑verified multi‑solution action annotations to provide privileged teacher guidance at each state, and Multi‑Solution Dual‑Level GRPO, which combines action‑level supervision with history‑conditioned planning‑quality supervision while handling multiple valid actions. The authors also present a scalable data generation pipeline and the Lite‑Dataset, enabling extensive evaluation across GUI benchmarks and demonstrating substantial performance gains over existing training paradigms.
By Yubin Wu, Zicheng Cai, Liping Ning, Hua Wang, Zhi Chen, Yaohua Tang, Hao Chen
The paper introduces a leakage‑aware framework for conversational task disambiguation over tabular data, defining ambiguous verifiable tasks and separating the agent into an asking policy and a solution policy. It presents AmbiTab, a benchmark suite that unifies six ambiguous datasets under a common representation, enabling evaluation of clarification strategies and oracle leakage. The authors train an asking policy with reinforcement learning, achieving improved disambiguation metrics across all datasets and higher task success on five of them, while providing diagnostics to measure oracle leakage during training.
By Nafiseh Ghoroghchian, Luis Scoccola, Tina Sedaghat, Omid Vaheb, Hannah Chen, Dino D'Agostino, Keyvan Golestan
The paper addresses policy optimization in online episodic tabular MDPs with adversarial losses and bandit feedback. By employing regularized Q‑functions, the authors eliminate the extra horizon factor present in previous policy‑optimization regret bounds, achieving high‑probability regret of ≥O(√{HS(H+A)T}) for known transitions and ≥O(HS√{AT}) for unknown transitions. These results improve horizon dependence and match the best-known bounds, with extensions to adversarial linear‑mixture MDPs.
By Mingyi Li, Taira Tsuchiya