The paper introduces an open midtraining recipe that primes base models for agentic post‑training, filling a gap between pretraining and full fine‑tuning. It trains on 256K context with hundreds of billions of tokens drawn from natural software‑engineering data and a small share of agentic traces. Applying this midtraining to Nemotron 3.5 Lightning and Super models yields measurable gains on SWE‑bench and DeepSWE benchmarks after identical full SFT, and improves robustness across agent harnesses.
By Soumye Singhal, Joseph Jennings, Ka Shun Shum, Alexander Bukharin, Khushi Bhardwaj, Zirui Liu, Venkat Srinivasan, Jonathan Li, Yash Jain, Kezhi Kong, Markus Kliegl, Olivier Delalleau, Sanjeev Satheesh, Mostofa Patwary
The paper introduces the Agentic Compositional Generalization hypothesis, suggesting that reinforcement learning (RL) primarily refines high‑level decision‑making behaviors that orchestrate pre‑trained low‑level skills, rather than teaching new domain‑specific skills from scratch. It proposes River, a training recipe that enhances reward quality by filtering low‑quality synthetic environments and adding process‑level behavior regularization. Using River, RL‑trained agents outperform other open‑source 8B models on four terminal‑agent benchmarks, achieving significant gains with fewer than 30% of the training environments.
By Yihang Yao, Bo Pang, Xuan Phi Nguyen, Ding Zhao, Shafiq Joty, Semih Yavuz
arXiv:2606. 04272v1 Announce Type: new Abstract: The standard LLM training pipeline applies reinforcement learning (RL) only after pre-training and supervised fine-tuning (SFT).
By Rachit Bansal, Clara Mohri, Tian Qin, David Alvarez-Melis, Sham Kakade
UniToolCall introduces a unified framework for tool-use in large language model agents, standardizing toolset construction, dataset generation, and evaluation. The framework aggregates over 22,000 tools and creates a hybrid training corpus of more than 390,000 instances by combining ten public datasets with synthetically generated, structurally controlled trajectories. It models diverse interaction patterns—single‑hop vs. multi‑hop, single‑turn vs. multi‑turn, serial vs. parallel execution—and adds an Anchor Linkage mechanism to enforce cross‑turn dependencies, while converting seven public benchmarks into a common Query–Action–Observation–Answer format for fine‑grained evaluation.
By Yijuan Liang, Xinghao Chen, Yifan Ge, Ziyi Wu, Hao Wu, Changyu Zeng, Wei Xing, Xiaoyu Shen
arXiv:2606. 00135v1 Announce Type: cross Abstract: Tool-calling is a central component of modern large language model (LLM) agents, equipping them with skills beyond their parametric knowledge.
By Tong Liu, Cheng Qian, Matej Cief, Yuan He, Daniele Dan, Nikolaos Aletras, Gabriella Kazai
ToolCompass is a post‑training framework that improves how large language model agents explore new tools by organizing tool‑call representations according to shared functions. It models each function class as a von Mises–Fisher distribution, reducing variation within a function while increasing separation between different functions, thereby guiding exploration toward functionally similar unseen tools. Experiments on AppWorld and FTRL show consistent gains, with up to a 10.71‑percentage‑point improvement in out‑of‑distribution task success over vanilla post‑training and outperforming competitive baselines.
By Junlin Fang, Chong Zhang, Do Nguyen-Thanh, Xiaogang Xu, Zhen Fang, Sean Du
arXiv:2608. 20314v1 Announce Type: new Abstract: Mid-training is increasingly recognized as a critical stage for shaping the capabilities of large language models.
By Fengqing Jiang, Yite Wang, Boyi Liu, Zhaoyang Wang, Canwen Xu, Zhewei Yao, Radha Poovendran, Yuxiong He
arXiv:2607. 10891v1 Announce Type: new Abstract: Large language models (LLMs) are rapidly shifting toward agents that solve tasks through diverse interfaces, including web and graphical user interfaces (GUIs).
By Qijia Shen, Zhiqi Huang, Vamsidhar Kamanuru, Aznaur Aliev, Jay Rainton, Ahmed Awelkair, Zhichen Zeng, Jiajun Li, Shi Dong, Yueming Yuan, Boyuan Ma, Qizheng Zhang, Jiwei Fu, Yuzhen Mao, Wendong Fan, Ping Nie, Philip Torr, Bernard Ghanem, Changran Hu, Jonathan Lingjie Li, Urmish Thakker, Guohao Li
arXiv:2607. 01232v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a central component of post-training large language models (LLMs), yet little is understood about how RL adaptation is distributed across transformer layers.
By Zijian Zhang, Rizhen Hu, Athanasios Glentis, Dawei Li, Chung-Yiu Yau, Hongzhou Lin, Mingyi Hong
PG-SFT: Balancing Capability Acquisition and Retention in Offline Agent Fine-Tuning explores how to maintain a model’s existing abilities while teaching it new ones through supervised fine‑tuning on offline agent trajectories. The authors compare standard SFT, KL‑penalty, and update‑magnitude constraints, finding that these methods still degrade non‑target capabilities. They introduce Privilege‑Guided SFT (PG‑SFT), which uses turn‑level information gain to modulate supervision strength, achieving a better trade‑off between acquiring new skills and preserving existing ones, though with a slight drop in target‑task performance.
By Ronghua Li, Zi Liang, Zhishan Li, Shinan Liu
arXiv:2606. 09932v1 Announce Type: cross Abstract: Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) has become a standard pipeline for Large Language Model (LLM) post-training.
By Runze Liu, Jiashun Liu, Xu Wan, Yuqian Fu, Ling Pan
arXiv:2610.19789v1 Announce Type: cross
Abstract: Self-distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published bas...
By Linhao Zhong, Bozhen Fang, Xuehao Wang, Duochao Shi, Zekai Luo, Guangming Yao, Yi Yuan, Hao Chen, Jingdong Chen, Chunhua Shen