The monograph "Statistical Foundations of Reinforcement Learning: A Non-Asymptotic Perspective" surveys recent advances in finite-sample statistical theory for reinforcement learning. It focuses on tabular Markov decision processes and examines three sampling settings—generative model, online, and offline RL—alongside model-based and model-free algorithmic paradigms. The work emphasizes minimax-optimal sample complexity, provides self-contained analyses, and briefly explores extensions such as robust, multi-agent, federated, and human-feedback RL.
By Yuxin Chen, Yuejie Chi, Jianqing Fan, Gen Li, Yuting Wei, Yuling Yan
The paper proposes ten desiderata for designing and evaluating reinforcement‑learning agents that learn continuously over a lifetime. It reviews existing approaches against these criteria, noting that no single agent has yet integrated all ten. Additionally, it introduces a compact persistent‑world scenario and a benchmark to test multiple behavioral capabilities within one continuous lifetime.
By Nis Meinert, Jakob Gawlikowski, Edward Grant, Max Cyprys, Nathan Schurr, Neil Boston, Alfredo Ibias, Milind Tambe, Janusz Marecki
Natural Language Actor-Critic (NLAC) is a new actor‑critic algorithm that trains large language model (LLM) agents by using a generative LLM critic to produce natural‑language feedback instead of scalar rewards. By providing richer, actionable explanations for suboptimal actions, NLAC helps LLM policies improve without relying on random exploration, and it can be trained off‑policy without policy gradients. Experiments on reasoning, web browsing, tool‑use, and dialogue tasks show that NLAC outperforms existing training methods and offers a more scalable and stable training paradigm for LLM agents.
By Joey Hong, Kang Liu, Zhan Ling, Jiecao Chen, Sergey Levine
PrivacyFlow introduces a new benchmark, PrivacyBench, and a defense system that separates privacy planning from task execution in autonomous LLM agents. The system traces data provenance, models recipient knowledge, and simulates inference attacks to weigh disclosure risk against task utility, allowing it to minimize or block data sharing. Experiments show that PrivacyFlow reduces privacy failures and increases completed task sequences by 22–55 percentage points, while revealing a tradeoff between privacy, utility, and user autonomy.
By Robin Staab, Tom Wollschl\"ager, Vahab Mirrokni, Reza Shokri
The paper introduces ORACLE, an online reasoning framework designed to anticipate smartphone scams early by analyzing partial, streaming app‑usage trajectories. It presents a real‑world benchmark of 12 scam types across 95 apps and 15‑day periods, and employs a self‑evolving context manager plus an on‑policy self‑distillation scheme to consolidate fragmented evidence and enhance early detection. Experiments demonstrate that ORACLE consistently improves early scam anticipation, providing timely warnings while reducing false alerts in realistic streaming scenarios.
By Wenbo Gao, Songbai Tan, Zhongan Wang, Fei Shen, Gang Xu, Huiping Zhuang, Yunyun Yang, Ming Li, Xiaofeng Zhu
Agentic World Modeling (AWM) redefines next-observation prediction as a multi-turn evidence-acquisition task, allowing agents to query a real environment safely and construct observations without mutating shared state. Integrated with trajectory-level test-time scaling, AWM enables parallel exploration over a single shared environment, outperforming single-shot simulation by 40.07% and trained world-model baselines by 12.36% on benchmarks such as τ-bench, AppWorld, and OfficeBench. This approach provides a practical foundation for safe and scalable agentic test-time scaling in shared, non-resettable environments.
By Wendi Li, Huseyin A. Inan, Wei-Ning Chen, Yanzhi Chen, Chetan Bansal, Saravan Rajmohan, Sharon Li
The paper investigates how tool-call traces from large language model agents can reveal user attributes. Using the PrivacyTrace benchmark, the authors show that an attacker can infer user profiles with a 26.25% success rate from tool-call data, outperforming prior guessing methods. The study highlights that tool arguments and results are key evidence, and linking traces across sessions increases profiling risk.
By Donghang Duan, Qingyu Liu, Yifei Liu, Na Zou, Xia Hu, Dongrui Liu
RetryRoute is a new routing framework for web agents that edit images by calling open editing models in a critique-and-retry loop. It addresses the problem that existing routers treat each retry as an independent draw, whereas empirical analysis shows that a retry after a failure succeeds only 17‑53% as often as a first try across six open models. By estimating each model’s chance of success given the request’s failure history, planning retries, escalations and abandonment, and incorporating live queue waits, RetryRoute improves success rates and delivery within service-level objectives compared to eleven prior baselines.
By Hanshuai Cui, Zhiqing Tang, Zhi Yao, Jun Luo, Weijia Jia
The paper introduces CEDA, a Centralized Training with Decentralized Execution algorithm that uses a Double Deep Q-Network and a QMIX-style mixing network to coordinate multiple drones for medical supply delivery during disaster response. CEDA incorporates a Priority-Preserving Fair Scheduling strategy that encodes triage priorities, fairness, and anti-starvation rewards, enabling drones to jointly optimize routing, coordination, and energy-efficient navigation under dynamic hazards. In simulations and PX4 SITL validation, CEDA achieves over 95% patient delivery, reduces obstacle collisions by more than 95%, and maintains high triage efficiency and fairness across patient classes.
By Aneesh Calyam, Subrahmanya Chandra Bhamidipati, Zack Murry, Sharan Srinivas
The paper introduces a new multi‑agent reinforcement learning framework that separates agent identity from behavior by using events to trigger behavioral changes. It proposes Neural Manifold Diversity (NMD) to create a continuous behavior manifold and an event‑based hypernetwork that generates Low‑Rank Adaptation (LoRA) modules for on‑the‑fly policy reconfiguration. The authors prove that this design preserves reward maximization and show empirically that it outperforms baselines, generalizes zero‑shot, and uniquely solves tasks requiring sequential behavior reassignment.
By Hannes B\"uchi, Manon Flageat, Eduardo Sebasti\'an, Amanda Prorok
TraceSpec is a framework that links trajectory evaluation with adaptive draft training to improve speculative decoding in large language model agents. It introduces a dynamic execution strategy combined with fixed‑prefix replay and provides a new corpus (TraceSpec‑Traj) of 10,000 training and 2,500 test states covering browsing and software engineering tasks. Experiments show that TraceSpec’s state‑conditioned draft allocation boosts replay throughput by 12.5%–22.6% and increases accepted length by up to 10.0% compared to fixed‑turn supervision.
By Jihao Zhao, Shuaishuai Zu, Pengfei Li, Biao Qin, Guanghuan Fang, Ben Feng, Baochuan Yang
The paper introduces KEEPER, a framework for learning control‑sufficient textual memory in long‑horizon large language model agents. It examines four auxiliary objectives that balance information content and update sensitivity, finding that reward‑related prediction yields the best task performance before and after offline reinforcement learning. A claim‑level analysis shows that incorrect claims can persist in memory even when corrective observations are made, highlighting the need to assess error persistence and valid‑information retention alongside task outcomes.
By Jinkyeong Choi, Chaebin Jeong, Junsung Jeon, Donghyeon Park
Saidzoda‑Tajik‑LFM‑2.6B is a 2.6‑billion‑parameter hybrid model adapted for Tajik tool calling. The adaptation uses two stages: LoRA continued pretraining on 1.49 million cleaned Tajik web documents followed by one epoch of supervised fine‑tuning on machine‑translated Tajik tool calls, English tool calls, and Tajik instructions. On a 174‑item Tajik tool‑calling test set, strict full‑call accuracy rises from 20.69 % for the base model to 64.14 %, while English performance remains unchanged; however, continued pretraining removes the native tool‑call format and part of the skill, and the model shows limited general assistant capabilities.
By Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon Shodyeva
SimXAI is a lightweight, LLM‑centric conversational layer that replaces traditional intent recognition, dialogue management, and template‑based response modules in GUS‑style ConvXAI agents. By using a single tool‑calling LLM to interface with existing XAI resources, it provides a modular interface that improves function‑selection accuracy, faithfulness, and handling of context‑dependent requests. In a user study, substituting SimXAI for the conversational layer of the TalkToModel agent led to more accurate task completion in fewer turns, demonstrating enhanced conversational quality while preserving faithfulness.
By Van Bach Nguyen, Qianli Wang, Nils Feldhus, J\"org Schl\"otterer, Christin Seifert
Capability‑Grounded Agentic Planning (CGAP) is a method that transforms a natural‑language request into a deterministic PDDL model by compiling a catalog of tools, then uses a classical planner to generate an abstract plan. An LLM interprets the request as goals and, during execution, adds missing entity‑specific calls while a verifier ensures each result satisfies the original request. Experiments on ToolSandbox, ComplexFuncBench, BFCL multi‑turn, and τ²‑bench retail show CGAP can outperform standard tool‑calling agents by up to 28.9 points.
By Shikha Shrestha, William Yeoh, Ning Zhang, Yevgeniy Vorobeychik
Hybrid Computer Use (HCU) is a unified interface that enables general agents to interact with computers through three complementary channels: programmatic code and command-line tools, semantic accessibility (AX) features, and visual graphical user interfaces (GUI). The design emphasizes efficient AX interaction, batch execution, and AX diffs to reduce round trips, while a supplementary skill guides agents in selecting and coordinating these channels. On the OSWorld 2.0 benchmark, HCU outperforms other systems in both efficiency and task success, demonstrating the benefits of flexible, multi‑channel computer‑use interfaces.
By Haoqing Wang, Xiaoyi Zhang, Sicheng Fan, Zhexin Zhang, Xinjie Zhang, Jiahao Li, Bin Li, Yan Lu
The paper evaluates six retrieval structures for multi‑tool requests in LLM agents, measuring how many required tools appear in the top‑k results and the associated LLM calls, tokens, and latency. Across two testbeds—345 AppWorld queries and 103 MCP‑Bench tasks—none of the untrained, single‑LLM‑call structures completes more than 10% of AppWorld queries at k = 16. A trained set‑level retriever and an LLM‑navigated hierarchy achieve higher completeness (up to 58% on MCP‑Bench) but at different costs and failure modes, with complete retrieval sets leading to 100% task success in a downstream AppWorld agent test.
By Shuyang Zhang
AgentCollab introduces a self-evaluation-driven collaboration paradigm for large language model agents, dynamically coordinating models of varying reasoning capacities during task execution. By using the agent’s own self-reflection to decide when to elevate to a stronger model, it avoids reliance on external routing modules and allocates additional reasoning budget only when recent failures indicate difficulty. Experiments on multi-step agent benchmarks demonstrate that this two-level small-large model setup consistently improves the accuracy-efficiency Pareto frontier of LLM agents.
By Wenbo Gao, Renxi Liu, Xian Wang, Fang Guo, Shuai Yang, Xi Chen, Hui-Ling Zhen, Hanting Chen, Weizhe Lin, Xiaosong Li, Yaoyuan Wang
The paper "SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directions" presents a unified framework for understanding autonomous RAG systems. It formalizes agentic retrieval-generation loops as finite-horizon partially observable Markov decision processes, introduces a taxonomy and modular architecture decomposition, and critiques current evaluation practices while highlighting systemic risks such as hallucination propagation and memory poisoning. The authors also propose doctoral-scale research directions for building reliable, controllable, and scalable agentic retrieval systems.
By Saroj Mishra, Suman Niroula, Umesh Yadav, Dilip Thakur, Srijan Gyawali, Shiva Gaire
DyCraft is an agentic system that generates long‑horizon 4D animated 3D scenes by decomposing the task into a detailed plan, a static scene, and a 3D storyboard of key poses. Each storyboard frame is a static scene that the agent can build and debug independently, allowing local motion to be authored with verified start and end states. In evaluations on 24 multi‑shot animation tasks, DyCraft reduces physical violations and produces higher‑quality scenes that are more often preferred by human raters compared to native GPT‑6 agents and prior baselines.
By Jeff Tan, Hang Dai, Zeqian Long, Yanzhe Lyu, R. Kenny Jones, Shangzhe Wu, Chen Geng, Jiajun Wu