MemComposer is a memory composer for large language model agents that selects, orders, and limits memories based on the current query and retrieved candidates. It is trained with Experience Policy Optimization (ExPO), a GRPO-based framework that evaluates assembled memory contexts through full agent interactions and optimizes the assembly policy using task‑success feedback. In experiments on three τ²‑bench domains, MemComposer outperforms the strongest baseline for each executor while reducing memory‑token usage by 16.9% and 4.9%, and its learned policy transfers to unseen domains, executors, and memory banks.
By Yixuan Xie, Xueqiang Xu, Xinyi Fan, Xinxi Lyu, Tao Feng, Xing Fan, Chenlei Guo, Jie Hao, Shaowen Wang, Jiashuo Sun, Jiawei Han
The study examines how temporal resampling (binning) of irregular clinical data affects the safety and performance of offline reinforcement learning (ORL) models in healthcare. By analyzing MIMIC‑IV insulin infusion records, the authors show that binning can reverse causal order and obscure critical events. In simulated trials with virtual type‑1 diabetes patients, ORL models trained on binned data performed up to 60% worse than those trained on unprocessed data, yet retrospective evaluation on resampled data overestimated their returns.
By Thomas Frost, Hrisheekesh Vaidya, Steve Harris
The paper introduces Recursive Reward Improvement (RRI), a method that enhances recursive self‑improvement loops by using an adversary to expose and fix weaknesses in model‑generated rubrics for open‑ended tasks. In RRI, an adversary crafts a rubric‑gaming answer, an honest answer is generated without rubric exposure, and up to two repair steps adjust the rubric so the honest answer scores higher. Experiments on HealthBench Professional show that RRI boosts adjusted scores by up to 12.4 points on a 27B model and yields significant gains on difficult and writing tasks, while also improving performance on unseen benchmarks PRBench Hard and ProfBench.
By Wei Dai, Qianchu Liu, Guanghui Qin, Xing Han, Tristan Naumann, Paul Pu Liang, Sheng Zhang
The paper introduces CA‑MAMBA, a constrained multi‑agent reinforcement learning method that predicts cost within a world model, enabling most policy updates to occur in imagination. Compared to a strong model‑free baseline, CA‑MAMBA requires 10–20 times fewer interactions on the median SMAC configuration and achieves 2.8–12 times higher MAMuJoCo rewards for the same budget. In controlled experiments, aligning the model’s cost estimates with a cost‑blind learner reduces that learner’s cost by more than half and satisfies a strict continuous‑control limit, indicating that accurate cost estimation is not the limiting factor.
By Raz Shmueli, Shahaf S. Shperberg
The paper introduces SIF, a reinforcement learning technique that uses selective internal feedback to improve language models’ reasoning without external verifiers. SIF constructs comparative rewards by comparing token‑level self‑certainty across multiple windows and peers, rewarding only when both voting levels agree, and applies sample and group‑level filtering to decide which responses influence policy updates. Experiments on three backbone models across 12 mathematical and scientific benchmarks show that SIF’s final checkpoints outperform the best‑validation checkpoints of Intuitor and Vigor, achieving a 4.71‑point accuracy gain on Qwen2.5‑Math‑1.5B and maintaining accuracy during continued optimization with minimal test‑time rollouts.
By Xun Lu, Shumeng Yang, Xinrui Wang, Songjun Tu, Linjing Li, Daniel Dajun Zeng
The paper derives closed‑form bounds on the performance of a drone‑mesh interception system before any learning policy is applied. It shows that, due to a separation constraint, a mesh of agents can cover at most 90.69 % of a plane (74.05 % of a volume) and that optimal hexagonal packing achieves this limit. The authors validate these theoretical ceilings with Monte‑Carlo simulations and argue that such bounds should be reported alongside any learning‑based results.
By Dmytro Babarytskyi
Data World proposes a multi‑agent engine that generates synthetic data environments, tasks, and evaluation rubrics by first specifying the underlying world rather than curating real files. The system produces thousands of training tasks across diverse domains and enables independent audits of target recoverability, while human curation ensures realistic question selection. Benchmark results show that models trained on these synthesized tasks still face significant challenges, yet RL‑zero training on the synthetic data yields measurable performance gains over baseline models.
By Zhanli Li, Zeyuan Xu, Chaoxu Pang, Zeyuan Liu, Anyang Ji, Te Sun, Kongcheng Zhang, Wenbo Xu, Xubin Yue, Shipeng Luo, Junjie Zhao, Yingfan Hua, Jingqing Ruan, Zhongda Su, Feng Hong, Zhizhao Zeng, Ke Zeng, Xunliang Cai, Hanjia Ye
PROBE is an algorithm that trains language‑model auditors to probe deployed tool‑using agents for policy violations through realistic, multi‑turn, tool‑mediated dialogue. By framing auditing as a two‑player Markov game and using a composite reward that balances violation discovery, behavioral plausibility, conversation progress, and policy diversity, PROBE outperforms zero‑shot adversaries in uncovering broader and higher‑yield policy violations. The study compares GRPO, DPO, and GFlowNet optimizations, demonstrating that multi‑turn behavioral auditing is a distinct and necessary axis of agent evaluation, complementing existing capability and safety benchmarks.
By Kshitij Mishra, Abhijith Sharma, Nils Lukas, Salem Lahlou
X‑RIC is a new framework that embeds agentic AI into O‑RAN architectures to enhance the RAN Intelligent Controller (RIC). It combines reactive reinforcement learning with strategic reasoning via automated behavioral cloning, a telecom language model, and closed‑loop co‑adaptation, and introduces BARX, a benchmark for evaluating agentic control. In simulations and testbed deployments, X‑RIC cuts operational costs by 89 %, meets SLA compliance, and converges 75 % faster with fewer training steps and lower variance.
By Adhwaa Al-Chaab, Ayman Younis, Dario Pompili
KnowGPO introduces a reinforcement learning framework that groups search agent trajectories by the task-relevant knowledge they acquire, rather than by surface observations. By decomposing required information into atomic knowledge conditions and using the incremental knowledge gain as a process reward, KnowGPO provides finer-grained credit assignment for deep search tasks. Experiments on Xbench, BrowseComp-ZH, and GAIA demonstrate that KnowGPO outperforms outcome-only and process-supervision baselines by 3.8 to 4.2 points, with an 8B model rivaling or exceeding most 32B open-source systems.
By Siqi Chen, Zhengxi Lu, Yuchen Yan, Xingyu Wu, Qipeng Chen, Xin Zhang, Aiting Liu, Chao Deng, Jie Liu, Jin Ma, Weiming Lu, Jun Xiao, Yongliang Shen
The paper introduces new methods for deciding which self‑evolving language model updates to keep. HeirCommit freezes each updated model and evaluates it on fresh questions before deciding to inherit it, while EvoHeir separates parameter inheritance from experience inheritance to repair failures and preserve successes. These techniques reduce harmful updates and improve performance on mathematics and general‑reasoning tasks compared to existing loops such as R‑Zero, Agent0, and Skill Self‑Play.
By Haoyi Yang, Kshitij Mishra, Nils Lukas, Salem Lahlou
The paper introduces Librarian Agents, a framework that trains large language models to proactively curate long-context workspaces by reorganizing raw documents into a hierarchical filesystem of synthetic files and reference links. This builder agent, optimized via reinforcement learning, aims to maximize downstream answering accuracy by anticipating future queries rather than reacting to them. Experiments show that Librarian Agents outperform leading retrieval and precomputation methods, achieving a 33‑point accuracy gain on BrowseComp‑Plus and a 31‑point improvement on BRIGHT in zero‑shot settings.
By Anikait Singh, Teresa Zhang, Yoonho Lee, Roshen Sanjay Nair, Aviral Kumar, Chelsea Finn
AVC-Reward introduces an event‑centric framework for evaluating audio‑video consistency, breaking down scenes into individual audio and visual events and assessing them at three levels: global narrative harmony, event‑level temporal synchronization, and fine‑grained attribute alignment. The metric, instantiated as a trainable reward model, is validated against human judgments using the AVC‑Benchmark, which contains diverse positive and negative samples from various generation models. AVC‑Reward outperforms existing automatic methods, offers interpretable failure localization, and enhances consistency when used as a reinforcement‑learning signal.
By Yuruo Wu, Kaihang Pan, Kai Shen, Jiacheng Li, Shuyuan Tu, Jisheng Dang, Li Tuoyu, Juncheng Li, Siliang Tang
The paper proposes a reinforcement learning framework that emphasizes designing feedback environments rather than solely relying on rewards. It integrates human domain knowledge via rubrics and model reasoning with hierarchical rewards to evaluate complex, partially successful outputs in open‑ended tasks. Applied to interactive web application generation, the model achieves 41.25% pass rate on MiniAppBench and 76.19% on ArtifactsBench, with ablation studies confirming the value of each component.
By Rui Jin, Jikai Chen, Chengyue Yu, Yihan Chen, Hao Zhou, Demin Zhu, Kaichen Yang, Dong Wang, Linjian Mo, Chenyi Zhuang
The paper introduces Tail‑Aware Credit Calibration (TACO), a method that adjusts the positive credit given to tokens in reinforcement learning for large language models (LLMs) by evaluating each token’s rarity relative to the policy’s expectations. TACO reduces credit for high tail‑risk tokens while still allowing them to influence learning, thereby mitigating the positive‑credit contamination problem seen in GRPO‑style methods. Experiments on four LLMs across eight reasoning benchmarks demonstrate that TACO consistently outperforms GRPO baselines and improves training stability for long‑horizon RL tasks.
By Xiuyi Lou, Zicheng Xu, Yu-Neng Chuang, Hoang Anh Duy Le, Zhaozhuo Xu, Guanchu Wang, Vladimir Braverman
The paper proposes a framework for service providers to display probabilistic service level information at booking time, aiming to inform customers and redistribute demand. It models the challenge as a sequential decision process, introduces recursive definitions of truthful probabilities, and imposes transparency constraints to limit manipulation. Using reinforcement learning and a sampling‑based mixed‑integer program, the authors approximate truthful probabilities and optimize the communicated information, demonstrating a trade‑off between individual transparency and overall customer utility in out‑of‑home delivery scenarios.
By Daniela Sailer, Robert Klein, Claudius Steinhardt
The paper studies a remote inference system that uses multimodal machine learning models to perform real‑time inference from features collected by remote sensors. It formulates a scheduling problem to minimize inference error, modeling error as a function of the Age of Information (AoI) vector, and solves it via a semi‑Markov decision process. The authors derive optimal policies for two‑modality and general multi‑modality cases, and propose low‑complexity alternatives that trade off speed and error.
By Keyuan Zhang, Yin Sun, Bo Ji
DeFlow is a decoupled offline reinforcement learning framework that uses flow matching to accurately model complex behavior manifolds. Instead of backpropagating through ODE solvers, it learns a lightweight refinement module within a data‑derived trust region of the flow manifold, preserving iterative expressivity while avoiding solver differentiation. The approach eliminates the need for balancing loss terms and achieves superior performance on the OGBench benchmark, also enabling efficient offline‑to‑online adaptation.
By Zhancun Mu, Chi Zhang
RoboAtlas is a contextual Active SLAM framework that balances geometric exploration with semantic reasoning through a scalable 3D semantic mapping system called OpenRoboVox. It uses frontier exploration, global semantic‑map reasoning, and egocentric VLM‑based reasoning guided by a contextual multi‑armed bandit to shift from exploration to semantically guided navigation as scene understanding improves. In simulation and on a Unitree Go2 robot, RoboAtlas achieved a 100% task success rate in environments over 1800 m² with ~30k semantic instances, and outperformed prior baselines on the GOAT‑Bench "Val Unseen" benchmark, reaching 90.6% SR with GPT‑4o and 88.8% SR with Qwen2.5‑VL‑7B.
By Alexander Schperberg, Shivam K. Panda, Abraham P. Vinod, Bhagawan Rokaha, M. K. Jawed, Stefano Di Cairano
The paper investigates how to infer agents’ underlying incentives in strategic multi‑agent settings, where traditional inverse reinforcement learning (IRL) may fail to produce rewards that guide optimal play. It introduces the concept of strategic identifiability and shows that learning from a fixed pair of policies can be insufficient regardless of sample size. To address this, the authors propose an active IRL algorithm for two‑player repeated Stackelberg games, combining maximum‑likelihood estimation with experimental design to achieve finite‑sample guarantees on sub‑optimality, and validate the approach in an anti‑poaching scenario.
By Antoine Bergerault, Cyrille Kone, Giorgia Ramponi