arXiv AI
Aug 3
PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization
arXiv:2605. 17877v2 Announce Type: replace Abstract: A significant hurdle for current LLMs is the execution of complex, multi-stage tasks.
Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agents
Related stories
arXiv Machine Learning
Jun 9
Reformulate LLM Reinforcement Learning for Efficient Training under Black-box Discrepancy
arXiv:2606. 08779v1 Announce Type: new Abstract: Reinforcement Learning (RL) has emerged as a pivotal post-training paradigm, yet it frequently suffers from unpredictable sub-optimum performance or even training collapses.
arXiv Machine Learning
5d ago
Steering Vectors as a Training Signal in LLM Post-Training
Steering vectors (SVs) are residual-stream directions that can shift LLM behavior when added to hidden states during inference. This study investigates whether the same SVs can guide parameter updates during post‑training, exploring supervised learning (SFT and distillation) and reinforcement learning with verifiable rewards (RLVR). The authors find that SV injection improves performance over plain SFT and distillation, achieving up to a 12% gain in certain distillation settings, and that applying a correctness‑contrast direction as advantage repair in RL recovers learning signals for held‑out math benchmarks. They also identify failure boundaries related to code generation, cross‑family layer transfer, and inference‑time application on SV‑trained checkpoints.
arXiv AI
Jul 7
LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL
arXiv:2607. 04412v1 Announce Type: new Abstract: Reinforcement learning (RL) for non-verifiable instruction following increasingly relies on LLM judges with prompt-specific rubrics as reward signals.
arXiv Machine Learning
Jun 25
FBOS-RL: Feedback-Driven Bi-Objective Synergistic Reinforcement Learning
arXiv:2605. 20256v2 Announce Type: replace Abstract: Reinforcement learning has become a cornerstone for aligning and unlocking the reasoning capabilities of large-scale models.
arXiv AI
Jul 17
Reward-Free Evolving Agents via Pairwise Validator
arXiv:2607. 14408v1 Announce Type: new Abstract: A self-evolving agentic loop repeatedly proposes a tweaked version of an agent (its prompt template or program) and accepts or rejects the change based on a per-iteration quality signal.
arXiv AI
Oct 2
ALER: Adaptive Learnable Experience Rewriting for Reinforcement Learning
arXiv:2610.00592v1 Announce Type: cross Abstract: In partially observable reinforcement learning (RL), a later observation can make stored information obsolete or change what it implies for the next...
arXiv AI
Oct 7
Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
The paper introduces SORL, a framework that stabilizes off‑policy reinforcement learning for long‑horizon large language model agents. It identifies two key instability sources—token‑level policy granularity mismatches and high‑variance off‑policy updates—and proposes turn‑level importance sampling and clipping‑triggered normalization to align optimization with multi‑turn interactions. Two instantiations, SO‑PPO and SO‑GRPO, are evaluated on open‑domain, multi‑hop, and medical QA benchmarks, as well as on asynchronous RL for mathematical reasoning, showing improved robustness without the need for early stopping or heuristic tuning.
arXiv Computation and Language
Sep 30
Dynamic Optimizations of LLM Ensembles with Two-Stage Reinforcement Learning Agents
arXiv:2502.04492v3 Announce Type: replace Abstract: The advancement of LLMs and their accessibility have triggered renewed interest in multi-agent reinforcement learning as robust and adaptive framew...
arXiv AI
Jun 3
Synthesize and Reward -- Reinforcement Learning for Multi-Step Tool Use in Live Environments
arXiv:2606. 03892v1 Announce Type: cross Abstract: Training LLMs to orchestrate multi-step tool calls is held back by three coupled obstacles: realistic stateful execution environments are costly to build, synthetic training queries are often detached from the server's actual state (so the generated tool calls fail to execute), and recall-based RL rewards incentivize verbose tool-calling patterns.
arXiv Machine Learning
Jul 31
Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale
arXiv:2607. 28074v1 Announce Type: cross Abstract: Computer-use agents learn from what their actions change, so training one needs applications it can act on, break and reset.
arXiv AI
Jul 29
PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments
arXiv:2605. 30880v4 Announce Type: replace-cross Abstract: World models for interactive text agents must typically be learned from observation-action trajectories alone.