AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,733 stories · RSS feed

arXiv Computation and Language
Oct 23

FusionEval: Benchmarking Model Fusion for Large Language Models

FusionEval is a new benchmark for evaluating model fusion techniques that combine multiple expert models into a single large language model. It provides controlled expert pools trained from Qwen3 and Llama on domains such as math, code, science, instruction following, and tool use, and evaluates capability retention, computational cost, runtime, and GPU memory across eight public benchmarks. Experiments show that training‑free fusion methods keep reasoning performance but hurt tool use, while MOPD maintains tool use and balances capabilities at the cost of extra data preparation and training.

By Shuo Cai, Yanggan Gu, Yuhang Liu, Su Lu, Yuanyi Wang, Yang Yu, Zhen Li, Zhitian Hou, Guanghao Zhu, Ming Li, Hongxia Yang
arXiv Computer Vision
Oct 23

Ask About the Right Moment: Training-Free Object Identity Reasoning for Video-Language Models

The paper introduces a training‑free method called “Track, Then Bind” for video‑language models to reason about object identity over time. It separates semantic identity binding from temporal identity maintenance by using a promptable video tracker to follow candidate objects and a frozen vision‑language model to bind an object to a track based on visible evidence. The approach achieves state‑of‑the‑art performance on several benchmarks, outperforming fine‑tuned trajectory models and other training‑free agents on tasks such as VET‑Bench, Perception Test cup games, and TOC‑Bench.

By Di Yang, Xiaochang Li, Muyang Liu, Yufei Wang, Yanhai Xiong
arXiv Computer Vision
Oct 23

MeSS: City Mesh-Guided Outdoor Scene Generation with Cross-View Consistent Diffusion

MeSS (Mesh-based Scene Synthesis) is a method that generates high‑quality, style‑consistent outdoor scenes using city mesh models as geometric priors. The pipeline first creates geometrically consistent sparse views with Cascaded Outpainting ControlNets, then densifies intermediate views via AGInpaint, and finally removes visual inconsistencies with GCAlign. Simultaneously, a 3D Gaussian Splatting scene is reconstructed from the mesh, enabling diverse rendering styles after synthesis.

By Xuyang Chen, Zhijun Zhai, Kaixuan Zhou, Zengmao Wang, Jianan He, Dong Wang, Yanfeng Zhang, Mingwei Sun, Xuqin Wang, R\"udiger Westermann, Tao Wu, Konrad Schindler, Liqiu Meng
arXiv Computation and Language
Oct 23

TauIndianBankBench: Procedure-Aware, Action-Oriented Evaluation of Tool-Using Conversational Agents

TauIndianBankBench is a new benchmark that evaluates conversational agents on 1,000 Indian retail‑banking tasks by checking not only the final records but also whether every required procedural step was performed and no unrequested changes were made. It highlights that models can appear successful when judged only on final records, yet may skip necessary steps or alter records incorrectly. The benchmark provides a more accurate assessment of language model performance in the Indian banking domain.

By Krishanu Adhikary, Prashant Devadiga
arXiv AI
Oct 23

SWE-DIVE: Multi-Turn Coding Agent Evaluation with Dynamic User Simulation

SWE-DIVE is a dynamic, interactive framework for evaluating coding agents in multi-turn scenarios. It introduces a user simulator that generates varied behavioral patterns and a conversation validator that checks whether the simulator’s outputs meet initial requirements, guiding the agent toward task completion. The framework runs within a Harbor environment that orchestrates the agent, simulator, and validator to assess success on coding tasks.

By Lorenz Brehme, Benjamin Steenhoek, Ruth Breu
arXiv Machine Learning
Oct 23

IRAH: Hierarchical Agentic Tree Search for Retrieval Pipeline Optimization

The paper introduces IRAH, a method for optimizing Retrieval-Augmented Generation pipelines by combining a Language Agent Tree Search for IR (LATS-IR) with offline meta‑learned initialization. LATS‑IR performs a UCT‑guided hierarchical search, first exploring macro‑structural pipeline choices and then refining numerical parameters, while using natural‑language reflections to guide mutations and provide inspectable search traces. Evaluations on three in‑distribution and one out‑of‑distribution benchmarks show that IRAH, especially when warm‑started, achieves the highest average held‑out nDCG@10 (0.4816) and outperforms TPE on two of the three in‑distribution benchmarks.

By Maria Ana Cardei, Aniket Singh, Amir Moghadam, Srividhya Baikadi, Rashmi Shetty
arXiv AI
Oct 23

Steering Black-Box Large Language Model Agents with Pitfall Anticipation World Model

The paper introduces the Pitfall Anticipation World Model (PAWM), a selective guidance system for black-box large language model agents. PAWM predicts imminent failures in a bounded multi-step future and intervenes only when a potential pitfall is detected, reducing costly and often uninformative step-by-step predictions. Trained first with supervised fine-tuning on synthetic pitfall-labeled data and then co-trained with a black-box LLM, PAWM boosts Claude Sonnet 4.5’s performance by 24.3% on ScienceWorld and 7.7% on WebArena-Lite, outperforming several baselines.

By Jiongxiao Wang, Han Ding, Youzhi Luo, Yawei Wang, Yun Zhou, Linbo Liu, Panpan Xu
arXiv AI
Oct 23

GenXR-Bench: Benchmarking Coding Agents for Interactive XR Development

GenXR-Bench is a new benchmark designed to evaluate coding agents in the context of interactive extended reality (XR) development. It comprises 112 expert-authored tasks—including feature implementations, bug repairs, and application generation—alongside a testing suite that uses scripted and adaptive interactions with programmatic and visual assertions. The benchmark achieved 96.5% accuracy against expert judgments on 124 candidate applications, yet current models only reach a 50.9% Pass@1 score, highlighting challenges such as asynchronous sensor lifecycles and embodied interaction semantics.

By William Huang, David Li, Xiang 'Anthony' Chen, Xun Qian, Nels Numan, Zhongyi Zhou, Hyunsung Cho, Salman Muin Kayser Chishti, Ding Xia, Xincheng Huang, Jiahao Ren, Faraz Faruqi, Tianyu Xu, Yang Zhang, David Kim, Ruofei Du
arXiv AI
Oct 23

RegoMender: A Multi-agent Framework for Automated Rego Policy Repair

RegoMender is a multi‑agent framework that automates the repair of Rego policies, a key component of Policy‑as‑Code in cloud‑native systems. It uses a shared blackboard and atomic toolkit to dynamically apply patching, rewriting, or rollback actions, achieving high syntactic (82.9%) and semantic (77.9%) success rates on the newly introduced RegoBench benchmark of 1,159 real‑world defects. The system also generalizes to other languages such as HCL and SQL, repairing 86.4% of 132 real‑world CVE vulnerabilities, thereby demonstrating robustness and practical security value.

By Ruijie Zhang, Huaxiao Yin, Li Hu, Fanqi Kong, Haozhe Liang, Shaoyin Cheng, Kun Zhang, Bibo Tu
arXiv AI
Oct 23

Allocating Indivisible Goods to Strategic Agents: Pure Nash Equilibria and Fairness

The paper studies how to allocate indivisible goods to strategic agents with additive valuations without monetary transfers. It investigates whether mechanisms can guarantee pure Nash equilibria while ensuring fairness, focusing on envy‑freeness up to one good (EF1) and envy‑freeness up to any good (EFX). The authors show that the Round‑Robin algorithm yields EF1 allocations at every pure Nash equilibrium, and that a cut‑and‑choose algorithm for two agents produces EFX and maximin share fair allocations in equilibrium, a property absent in the non‑strategic setting.

By Georgios Amanatidis, Georgios Birmpas, Federico Fusco, Philip Lazos, Stefano Leonardi, Rebecca Reiffenh\"auser
arXiv AI
Oct 23

Learning What to Remember: Resource-Efficient Self-Improvement for Lifelong Agents

The paper introduces TokenWorth, a lightweight framework that learns the value of past experiences for self-improving agents operating under strict resource constraints. By using sparse execution feedback, TokenWorth guides both retrieval and memory management without updating the core model, enabling agents to adaptively reuse only the most beneficial experiences. Experiments on LifelongAgentBench show that TokenWorth outperforms existing lifelong-memory methods in task success while minimizing output tokens within the given budgets.

By Guannan Lai, Zi-Yang Bai, Jun-Peng Jiang, Han-Jia Ye
arXiv AI
Oct 23

SUDP: Secret-Use Delegation Protocol for Agentic Systems

The paper introduces the Secret-Use Delegation Protocol (SUDP), a method for agentic systems to perform user-authorized operations without granting reusable authority to untrusted requesters. It formalizes the Agent Secret Use (ASU) problem, outlines seven required security properties, and demonstrates that SUDP satisfies six of them under standard cryptographic assumptions, with the remaining property addressed via hardware-rooted runtime composition. A reference implementation shows minimal overhead for mediated calls.

By Xiaohang Yu, Hejia Geng, Xinmeng Zeng, William Knottenbelt
arXiv Machine Learning
Oct 23

Lean Finder V2: Semantic Search for Mathlib That Disambiguates User Queries

Lean Finder V2 is a semantic search framework for Mathlib that resolves ambiguous user queries by combining adversarial training at the model level with input-level query rewriting. It addresses the common scenario where mathematicians and coding agents issue short, context‑poor queries, which existing search engines struggle to interpret. On the MathlibQR benchmark, Lean Finder V2 achieves at least an 11% relative improvement over prior engines and further boosts retrieval performance through optimized query rewriting, while also enhancing downstream theorem‑proving success when integrated with Claude Code.

By Jialin Lu, Xiaoyu Liu, Hengrui Luo, Rodrigo Stehling, Linxi Liu, Weiran Sun, Yuepeng Wang, Wuyang Chen
arXiv Machine Learning
Oct 23

Training Small Language Models for Transferable Tool Selection

The paper presents a method for training a small, open-source language model to select appropriate tools from large tool corpora, addressing the bottleneck of tool selection for LLM agents. By using reinforcement learning, the authors demonstrate that a Qwen3-8B model can outperform a much larger Llama-4-Maverick model on the ToolBench benchmark. The trained model also transfers to different retrievers and generalizes to new tool domains.

By Zhaoyi Zhou, Akshayaa Magesh, Jeongyeol Kwon, Artem Kuriksha, Daniel Jiang, Andrea Zanette, Ayush Jain
arXiv Machine Learning
Oct 23

TerraTransfer++: Demonstration-Free End-to-End Planner

TerraTransfer++ demonstrates a demonstration‑free, end‑to‑end autonomous driving framework that leverages self‑play in vectorized simulators to pretrain a policy. The method distills self‑play rollouts into a flow‑matching trajectory teacher and trains lightweight adapters on a frozen vision backbone, using action KL divergence, a batch‑relational low‑rank loss, and trajectory‑matching loss. The resulting closed‑loop policy surpasses all published imitation‑based methods on the HUGSim benchmark, while the planner achieves state‑of‑the‑art performance on NAVSIM navhard and navtest without any human demonstrations.

By Zikang Xiong, Weixin Li, Zhouchonghao Wu, Akshay Rangesh, Saarth Bonde, Grantland Hall, Chen Tang, Yihan Hu, Wei Zhan
arXiv Machine Learning
Oct 23

A Transformer-based Approach for Real-time Multi-agent Path Planning with Varying Initial/Final Conditions

The paper introduces OCFormer, a transformer-based neural network for real‑time multi‑agent path planning. It maps a simple latent space where agents evolve independently to the physical space of interacting agents, using a physics‑informed loss that incorporates the Hamiltonian ODE residual and boundary matching. Trained on a distribution of boundary conditions, OCFormer produces near‑optimal trajectories for unseen configurations in about 220 ms and can be adapted to out‑of‑distribution scenarios via lightweight transfer learning.

By Qian Zhang, Shanqing Liu, Jerome Darbon, George Karniadakis
arXiv Machine Learning
Oct 23

Reliable AI-Assisted Decisions via Selective Auditing

The paper introduces a framework for online monitoring of AI systems that produce decisions with associated confidence scores. It formulates the problem as an online constrained optimization task, aiming to minimize expert auditing while ensuring that the proportion of unaudited AI errors remains below a user‑defined threshold. The authors present a family of algorithms with provable error‑control guarantees, including an optimally efficient method that achieves the best possible auditing rate for both adversarial and i.i.d. data streams, and demonstrate its effectiveness on tasks such as screening harmful requests, text annotation, and data labeling.

By Isaac Gibbs, Elise L. Han, Ryan J. Tibshirani, Tijana Zrnic
arXiv Machine Learning
Oct 23

Super Agents and Confounders: Influence of surrounding agents on vehicle trajectory prediction

The paper analyzes state‑of‑the‑art vehicle trajectory predictors and finds that many surrounding agents actually worsen prediction accuracy. Using Shapley attribution, it shows models learn unstable, non‑causal decision schemes that vary across training runs. The authors introduce a Conditional Information Bottleneck that compresses agent features and ignores unhelpful ones, improving accuracy and robustness across datasets and architectures.

By Daniel Jost, Luca Paparusso, Martin Stoll, J\"org Wagner, Raghu Rajan, Joschka B\"odecker
arXiv Machine Learning
Oct 23

Bayesian Ambiguity Contraction-based Adaptive Robust Markov Decision Processes for Adversarial Surveillance Missions

The paper introduces an adaptive Robust Markov Decision Process framework for autonomous Intelligence, Surveillance, and Reconnaissance missions using Collaborative Combat Aircraft. It models adversarial uncertainty with finite threat prototypes and refines policies by eliminating inconsistent threat models, enabling a shift from worst‑case protection to more efficient mission strategies while preserving robustness. Experiments demonstrate higher mission rewards and fewer exposure events compared to nominal and static robust planners across various network topologies.

By Jimin Choi, Max Z. Li
arXiv Machine Learning
Oct 23

High-dimensional learning dynamics of multi-pass Stochastic Gradient Descent in multi-index models

The paper presents an asymptotically exact description of the learning dynamics of multi‑pass stochastic gradient descent (SGD) in high‑dimensional non‑convex multi‑index models. By considering sub‑linear batch sizes and a critical learning‑rate scaling, the authors derive a continuous‑time scaling limit governed by dynamical mean‑field equations driven by a scalar Poisson jump process. This framework unifies several existing theories of SGD—gradient flow on empirical risk, autonomous ODEs for online SGD, and diffusion approximations in least‑squares—while revealing distinct behavior for general multi‑pass SGD beyond these special cases.

By Zhou Fan, Leda Wang