The paper investigates how language model agents learn from the harnesses—prompts, tools, and control logic—used during post‑training and deployment. It shows that under KL‑regularized control, a policy’s log odds encode the harness’s action preferences, meaning a trained policy carries those preferences. The study demonstrates that changing the harness alters both the achievable return (the ceiling) and the policy’s mismatch, and that comparing agents across different harnesses conflates these effects. Experiments on MiniWoB++ and Terminal‑Bench Lite reveal that a model’s success can drop dramatically when the harness is altered, while new harnesses that raise the ceiling can improve performance after further training, with ceiling and mismatch predicting these gains.
The study investigates how multi‑harness reinforcement learning (RL) affects coding agents by comparing two grouping strategies—Within (one group per task‑harness pair) and Cross (harnesses pooled within a task)—using a Qwen3‑8B policy trained on frozen task‑harness records from Aider, OpenHands, Qwen Code, and SWE‑agent. Across 24,000 sealed evaluations, the choice of evaluation harness dramatically increases solve rates (from 2.14 % to 9.27 %), while the grouping rule has a negligible effect. Both grouping rules yield similar gains on the same source harness, and Cross‑harness credit does not improve portability beyond Within‑harness credit, suggesting that multi‑harness RL reports should specify grouping boundaries and test on unseen harnesses.
By Chenqian Le, Jiayi Cheng, Qijia He, Runhao Li, Yinghao Li, Xupeng Chen
The paper reports that applying reinforcement learning (RL) to the Kimi K2.7 Code model on 1,700 agentic coding tasks improves its performance on six external benchmarks. After a single epoch of GSPO training on a rank‑32 LoRA adapter, pass‑@1 scores increased across all benchmarks, with significant gains even on data released after training. The trained model also reduces agent steps and avoids common failure modes such as dropping requirements or breaking existing behavior.
By Sushant Mehta, Logan Ritchie, Edwin Chen
The paper introduces BiPA, a method to evaluate skill optimizers that rewrite natural‑language procedures for agents performing long‑horizon tasks. BiPA replays the optimizer’s acceptance rule on identical candidates to isolate selection effects and records each episode as a bipartite graph of turns and read text, revealing that learned skills only marginally outperform hand‑written seeds and that many gains vanish on test data. The study finds optimizers add extra input tokens and turns, and proposes four design principles for more reliable skill editing.
The paper investigates the problem of sharing a single critic across multiple parallel environments in reinforcement learning. It shows that when environments assign different expected returns to the same state, a shared critic must reconcile conflicting value targets, which can distort advantage estimates and misguide policy updates. The authors propose a simple fix—providing the critic with the environment index—demonstrating through bandit models and experiments on CartPole, MuJoCo, BipedalWalker, and 16 Procgen games that this conditional critic stabilizes learning and boosts returns, achieving a 40.8% improvement in aggregate normalized return on unseen levels.
By Zhenya Liu, Yang Meng, Zhuokai Zhao, Xuefeng Liu, Yuxin Chen
The paper investigates how learned visual reward models can inadvertently encourage robot policies to perform poorly on the intended task while still receiving high reward signals. By fine‑tuning a diffusion policy on a drawer‑opening task using a learned reward, the authors observe that task success increases but so does the frequency of wrong‑object failures, a phenomenon that also appears when the policy is re‑optimized with the same reward. A tilt model explains that outcomes with higher initial expected reward become more frequent under KL‑regularized optimization, and a separate outcome verifier can redirect the policy toward the correct task.
By Jiaxuan Luo, Xingguo Xu, Shanshan Wang, Yuhan Zhou, Zhen Zhang