arXiv:2606. 00017v1 Announce Type: new Abstract: Training language model agents for multi-agent strategic interaction presents a core difficulty: the quality of any action may depend on future events that never materialize, on moves that violate game rules, or on decisions made by other players.
arXiv:2607. 04470v1 Announce Type: cross Abstract: Large Language Models (LLMs) offer a natural interface for translating human objectives into reward signals for cooperative multi-agent reinforcement learning (MARL), yet the training-time dynamics of this integration remain poorly understood.
arXiv:2606. 19990v1 Announce Type: new Abstract: While RL has become a promising tool for refining world models, existing methods largely rely on conservative rollouts near the training distribution, limiting exploration, behavioral diversity, and richer dynamic discovery.
Reward Engine is a method for creating a reward function for open‑ended tool agents by first determining what success looks like and then optimizing toward it. It builds a RewardPack of test cases and rubrics from initial trajectories, which is frozen and used to guide the agent’s output selection and can also serve as a reinforcement‑learning reward. Experiments on WorkBuddy Bench Office and Claw‑Eval show that the constructed reward aligns with benchmark graders on most candidate pairs and improves three‑run consistency from 45.0% to 54.0% and from 78.4% to 83.7%.
The paper proposes CANOPY, a minimalist reinforcement learning protocol that addresses two common pitfalls—signal starvation and policy drift—in outcome‑only RL for long‑horizon interactive tasks. By scaling same‑task exploration, keeping updates on‑policy, and anchoring updates with KL divergence, CANOPY enables a Qwen3‑14B agent to achieve top leaderboard results on the AppWorld coding benchmark without auxiliary supervision or elaborate scaffolding. The approach also improves performance on SWE‑bench for a Qwen3.5‑9B model.
arXiv:2610.19868v1 Announce Type: cross Abstract: Training LLM agents on long-horizon interactive tasks with reinforcement learning typically relies on human-annotated tasks and reliable verification...
arXiv:2606. 17043v1 Announce Type: cross Abstract: When pretrained VLA policies are fine-tuned through online RL, each rollout episode produces only a single binary outcome (success or failure), yet the actor update requires per-transition supervision.
The paper introduces T1, a 122‑billion‑parameter Mixture‑of‑Experts model trained with reinforcement learning to perform long‑horizon terminal tasks such as coding and scientific discovery. T1 operates a real shell in a cloud sandbox, making over 300 tool‑call turns per task and receiving rewards from task‑specific verifiers. The authors detail a training recipe that includes aggressive warm‑starting, TITO construction with drift repair, and rollout‑routing replay, achieving significant performance gains on Terminal‑Bench 2.1 and surpassing GPT‑5.4 and GLM‑5.1 on the Long‑Horizon Terminal Bench.
arXiv:2605. 17877v2 Announce Type: replace Abstract: A significant hurdle for current LLMs is the execution of complex, multi-stage tasks.
StraTA introduces Strategic Trajectory Abstraction, a framework that samples a compact strategy from the initial task state and conditions subsequent actions on that strategy, training strategy generation and action execution jointly with a hierarchical GRPO-style rollout design. The method enhances exploration and credit assignment over long horizons by incorporating diverse strategy rollouts and critical self-judgment. Experiments on ALFWorld, WebShop, and SciWorld demonstrate that StraTA consistently improves sample efficiency and final performance, achieving success rates of 93.1% on ALFWorld, 84.2% on WebShop, and a 63.5% overall score on SciWorld, surpassing frontier closed‑source models.
Optimistic Policy Regularization (OPR) is a simple reward‑shaping technique that augments on‑policy agents with a closed‑form bonus derived from elite action priors estimated from top‑quartile episodes. By adding a bounded transform of the log‑ratio of this prior to the policy’s action distribution, OPR requires no extra networks or off‑policy corrections and can be implemented as a one‑line change to PPO’s reward. Experiments on 49 Atari games show that OPR achieves a median human‑normalized score of 148.4 % with only 10 M steps, outperforming published PPO on 36 of 49 games and surpassing self‑imitation learning when matched on frames and backbone.
AgentRM proposes a generalizable reward model to guide LLM-based agents during test-time search, outperforming direct policy fine-tuning. Three reward modeling strategies—explicit, implicit, and LLM-as-a-judge—are explored, and AgentRM improves base policy performance by an average of 8.8 points across nine tasks, surpassing top general agents by 4.0 points. It also shows strong weak-to-strong generalization and can boost specialized agents, with plans to release code for further research.