The paper investigates how language‑model agents learn from the harnesses—prompts, tools, and control logic—through which they are deployed. Using KL‑regularized control, it shows that a trained policy’s log odds reflect the harness’s action preferences, and that the agent’s deployed return equals the harness’s best possible return minus a mismatch term caused by the policy’s suboptimal choices. Experiments on MiniWoB++ and Terminal‑Bench Lite demonstrate that changing harnesses can drastically alter performance, and that the ceiling‑minus‑mismatch metric predicts gains better than either component alone.
By Jian Mu, Ruibo Shen, Yao Shu
The paper introduces BiPA, a method to evaluate skill optimizers that rewrite natural‑language procedures for agents performing long‑horizon tasks. BiPA replays the optimizer’s acceptance rule on identical candidates to isolate selection effects and records each episode as a bipartite graph of turns and read text, revealing that learned skills only marginally outperform hand‑written seeds and that many gains vanish on test data. The study finds optimizers add extra input tokens and turns, and proposes four design principles for more reliable skill editing.
The paper introduces TalkMesh, a decentralized network of small language model agents that learn to communicate effectively during inference. Each agent proposes an answer, scores it with a confidence head, and the most confident agent broadcasts a hint; lower‑confidence agents revise their proposals if a new suggestion scores higher. This gossip‑based consensus, trained via group relative policy optimization, enables a mesh of three agents to match the accuracy of majority voting over 32 samples, and scales to larger meshes to significantly boost performance on benchmarks like GSM8K and MATH-500.
By Mehmet Kerem Turkcan
The study investigates how multi‑harness reinforcement learning (RL) affects coding agents by comparing two grouping strategies—Within (one group per task‑harness pair) and Cross (harnesses pooled within a task)—using a Qwen3‑8B policy trained on frozen task‑harness records from Aider, OpenHands, Qwen Code, and SWE‑agent. Across 24,000 sealed evaluations, the choice of evaluation harness dramatically increases solve rates (from 2.14 % to 9.27 %), while the grouping rule has a negligible effect. Both grouping rules yield similar gains on the same source harness, and Cross‑harness credit does not improve portability beyond Within‑harness credit, suggesting that multi‑harness RL reports should specify grouping boundaries and test on unseen harnesses.
By Chenqian Le, Jiayi Cheng, Qijia He, Runhao Li, Yinghao Li, Xupeng Chen
The paper investigates how different estimators of the reverse Kullback–Leibler (KL) divergence used as a regularization term in reinforcement learning (RL) training of large language models (LLMs) affect training stability and downstream performance. By analyzing gradient bias across various estimator configurations, the authors demonstrate that biased gradients can cause training instabilities, while unbiased configurations improve performance on both in‑domain and out‑of‑domain tasks. Experiments on Qwen2.5‑7B, Llama‑3.1‑8B‑Instruct, and Qwen3‑4B‑Instruct‑2507 confirm these findings and show that KL regularization also stabilizes off‑policy RL training in asynchronous setups.
By Vedant Shah, Johan Obando-Ceron, Vineet Jain, Brian Bartoldson, Bhavya Kailkhura, Sarthak Mittal, Glen Berseth, Pablo Samuel Castro, Yoshua Bengio, Esmeralda S. Whitammer, Moksh Jain, Siddarth Venkatraman, Aaron Courville
The paper reports that applying reinforcement learning (RL) to the Kimi K2.7 Code model on 1,700 agentic coding tasks improves its performance on six external benchmarks. After a single epoch of GSPO training on a rank‑32 LoRA adapter, pass‑@1 scores increased across all benchmarks, with significant gains even on data released after training. The trained model also reduces agent steps and avoids common failure modes such as dropping requirements or breaking existing behavior.
By Sushant Mehta, Logan Ritchie, Edwin Chen
The paper investigates how learned visual reward models can inadvertently encourage robot policies to perform poorly on the intended task while still receiving high reward signals. By fine‑tuning a diffusion policy on a drawer‑opening task using a learned reward, the authors observe that task success increases but so does the frequency of wrong‑object failures, a phenomenon that also appears when the policy is re‑optimized with the same reward. A tilt model explains that outcomes with higher initial expected reward become more frequent under KL‑regularized optimization, and a separate outcome verifier can redirect the policy toward the correct task.
By Jiaxuan Luo, Xingguo Xu, Shanshan Wang, Yuhan Zhou, Zhen Zhang
arXiv:2610.08963v1 Announce Type: new
Abstract: Asynchronous reinforcement learning (RL) for large language model (LLM) agents trains one policy on trajectories generated by another: rollouts come fr...
By Yifan Zhang
Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow...
The paper introduces BiPA, a method for evaluating skill optimizers that rewrite natural‑language procedures for agents. By replaying the optimizer’s acceptance rule on identical candidates and recording episodes as bipartite graphs of turns and read text, BiPA isolates the effect of selection from the cost of added input tokens. Experiments on three optimizers, three benchmarks, and two models show that learned skills perform only marginally better than hand‑written seeds, and that apparent dev‑set gains often disappear on test, revealing that optimizers tend to add redundant rules that increase token usage.
By Jiawei Zhou, Bo Peng, Yuxuan Huang, Ming Zhang, Youbang Sun, Xiang Wang, Chaochao Lu
The study investigates whether adaptive layer-skipping and repetition programs truly improve language‑model inference or merely exploit input‑blind controls. Using 32 such programs on Qwen3‑4B‑Base and Llama‑3.1‑8B across 4,413 multiple‑choice items, the authors compare real program gains to those from input‑blind perturbations and fixed letter offsets. Results show that input‑blind controls can produce comparable or even larger headroom, and that real programs only outperform controls in specific menu configurations, indicating that the apparent benefits may not be due to the actual layer computation choices.
By Yibei Guo, Rui Liu
JevAgentHarness is a new framework that benchmarks, deploys, and trains decision models to share control with large language models (LLMs) in agentic tasks. It introduces AgentDecisionBench to evaluate decision models from isolated questions to full agent control, revealing that accurate isolated decisions still miss many tasks and that naïve delegation can reduce success. By fitting confidence thresholds per choice and training a 4B decision model on states after agent mistakes, JevAgentHarness achieves high task success while cutting cost and speeding up execution, with results and resources made publicly available.
By Kunlun Zhu, Tao Feng, Xu Han, Heng Wang, Ling Yang, Xiangru Tang, Jiaxuan You, Heng Ji