AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,750 stories · RSS feed

arXiv AI
Sep 12

What a Random Draw from the MCP Registry Contains, and What Tool-Use Benchmarks Contain Instead

The study examines the composition of a random sample from the Model Context Protocol (MCP) registry, revealing that only 48.8% of the 400 sampled npm/stdio servers successfully complete an initialization handshake, compared to 66.7% for a hand‑curated frame. Among the servers that run, there are no fatal JSON Schema violations across 2,766 advertised tools, but optional safety annotations vary widely, with a 58.8% omission rate in the random draw versus 41.5% in the curated set. The authors also compare MCP tool descriptions to two benchmark corpora, finding minimal near‑duplication in real MCP tools (2.8%) and significant repetition in synthetic datasets (up to 85.6%).

By Haseeb Mohammed Afsar
arXiv AI
Sep 12

BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure

BenchShield is a formal, model-backed instrumentation layer designed to protect reward integrity in large language model (LLM) agent benchmarks. It uses a finite lifecycle model of reward-relevant events to run a static, phase-aware taint analysis that flags potential reward-hacking paths before execution, and a runtime analysis that attributes concrete agent actions and provides evidence-backed claims. The system was evaluated on a corpus of 456 adjudicated trajectories from over 31,000 public agent runs across three benchmarks, showing significant improvements in recall, coverage, and cost efficiency compared to a baseline hackability scanner.

By Shenghan Zheng, Zonglin Di, Yimin Liu, Kyoung Whan Choe, Jiankai Sun, Heguang Lin, Penghao Jiang, Yifeng He, Xiao Cheng, Jicheng Wang, Wenbo Chen, Alex Yates, Yinzhe Zhao, Bingran You, Yuan Gao, Ayush Munot, Shubham Gaur, Zhe Ye, Hao Wang, Xiangyi Li, Dawn Song, Christophe Hauser
arXiv AI
Sep 12

terms.txt: A Consent and Compensation Protocol for Agentic Web Access

The paper introduces **terms.txt**, a new protocol that extends the traditional robots.txt format to allow website owners to specify per‑path, per‑purpose access rules for automated agents. It proposes an origin‑enforced exchange using Web Bot Auth signatures, signed intent, delegation tokens, HTTP 402 negotiation, and signed receipts to enforce, audit, and contract these rules. A lightweight implementation adds only 0.20 to 0.65 ms per request on a single vCPU.

By Rajarshi Chowdhury
arXiv AI
Sep 12

2AM: Grounding Agent-Side Memory as Guidance for Steerable Action Models in Long-Horizon Manipulation

The paper introduces 2AM, a system that separates memory and action execution in long‑horizon robot manipulation. 2AM stores task memory exclusively in a multimodal Agent, while a single RGB‑based, stateless Action Model performs motion based on language and optional 2D hints. On the LIBERO‑Mem benchmark, 2AM achieves 76.3% average completion—over 61 points higher than the best baseline—demonstrating that agent‑side memory and precise steering of the Action Model can substantially improve performance.

By Yutong Hu, Fengjiao Chen, Xuezhi Cao, Renaud Detry
arXiv AI
Sep 12

Agent-Integrated Software: Interaction Contracts and Continuous Assurance

The paper introduces Agent-Integrated Software (AIS), a pattern that embeds an intelligent agent within an existing application to address persistent coordination challenges. It defines Intent-Level Interaction Abstraction (IIA) as the task semantics for user inspection and control, and presents interaction contracts that constrain the relationship between AIS execution and IIA states. The authors provide a transition-system model, a compact disclosure contract, and conditional propositions to illustrate how local component validity is insufficient and how admission invariants can be separated from planning, concluding with a research agenda for making agent integration a maintainable software engineering discipline.

By Shengcheng Yu, Chunrong Fang, Zhenyu Chen
arXiv AI
Sep 12

Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learning

The paper reviews hierarchical reinforcement learning (HRL) as a method for enabling agents to explore, plan, and learn in complex, open-ended environments by uncovering temporal structure in experience streams. It discusses the unclear definition of what makes a structure useful, the benefits of HRL for decision‑making challenges, and its impact on AI agent performance trade‑offs. The authors survey various HRL methods—from online learning to offline datasets and large language model integration—and outline the challenges and suitable domains for temporal structure discovery.

By Martin Klissarov, Akhil Bagaria, Ziyan Luo, George Konidaris, Doina Precup, Marlos C. Machado
arXiv AI
Sep 12

An Agentic Evaluation Framework for AI-Generated Scientific Code in PETSc

The paper presents PETSCAgent-Bench, a multidimensional benchmark and agent-based framework designed to evaluate AI-generated scientific code that uses the PETSc library. It combines deterministic checks with LLM-based assessments across five categories—correctness, performance, code quality, algorithmic appropriateness, and library-specific conventions—using a 14-evaluator pipeline. The framework demonstrates that while current large language models produce readable, well-structured code, they often fail on correctness and library conventions in realistic PETSc problems, revealing gaps that simple pass/fail tests miss.

By Hong Zhang, Barry Smith, Satish Balay, Le Chen, Murat Keceli, Lois Curfman McInnes, Junchao Zhang
arXiv AI
Sep 12

DCO: Dynamic Cache Orchestration for LLM Accelerators through Predictive Management

The paper proposes DCO, a dynamic cache orchestration scheme for multi-core AI accelerators that uses application-aware policies and dataflow information to guide cache replacement, bypass decisions, and thrashing mitigation. Using a cycle-accurate simulator, the authors demonstrate up to 1.80× speedup over conventional cache architectures and validate the approach with an analytical model and RTL implementation. The design occupies 0.064 mm² on a 15 nm process and operates at 2 GHz, showing that a shared system-level cache can simplify programming while boosting performance for large language model workloads.

By Zhongchun Zhou, Chengtao Lai, Yuhang Gu, Wei Zhang
arXiv AI
Sep 12

Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

Sci‑MMR is a new benchmark for multi‑step evidence‑grounded scientific reasoning in multimodal agents, featuring 235 multi‑hop tasks across four disciplines and an average of nine figure panels per task. It evaluates not just final answer accuracy but also the recovery of structured evidence from scientific claims, citations, visual data, and supporting regions. Experiments on eight state‑of‑the‑art models show a gap of over 20 points between answer accuracy and complete evidence recovery, highlighting significant challenges in evidence acquisition and integration.

By Jiaqiang Li, Yajie Yang, Zhiheng Xi, Jiadong Chen, Enyu Zhou, Senjie Jin, Yang Nan, Jiazheng Zhang, Han Wang, Yanxin Li, Dingwei Zhu, Bicheng Deng, Yuhui Wang, Xiang Zheng, Qi Zhang, Lei Bai, Xingjun Ma, Tao Gui
arXiv AI
Sep 12

Autonomous Chemical Mechanistic Discovery through Agentic Reasoning and Validation

The paper introduces ARCHE, an autonomous system that combines a general-purpose reasoning model, a domain-specialized computational chemistry model, and a structured tool registry to automate chemical mechanism discovery. ARCHE interprets scientific questions, generates and prioritizes mechanistic hypotheses, orchestrates computational workflows, and refines conclusions in a closed loop. The authors validate the system on three challenging scenarios, including reconstructing stereocontrolling transition states, proposing a radical pathway for an unpublished reaction, and identifying a descriptor governing selectivity in nickel-catalyzed cross‑coupling reactions.

By Dong Li, Sixuan Mi, Zihao Ye, Huan Xiong, Tao XU, Tong Zhu, Aijia Zhang, Junqi Gao, Kaiyan Zhang, Shijie Wang, Bowen Zhou, Yuqiang Li, Biqing Qi
arXiv AI
Sep 12

Memory Compression for High-Fanout Agent Sandboxes

The paper introduces AgentZip, a memory compression system tailored for high‑fanout AI‑agent sandboxes. It exploits template‑relative and cross‑sandbox redundancy, expands compression to any profitable page, and shifts overhead control to restore‑time prefetching. The approach aligns compression with LLM waiting periods, achieving up to 8.7× memory reduction and reducing slowdown to 1.40× while preserving most savings.

By Mengming Li, Ceyu XU, Qijun Zhang, Jiangnan Yu, Xiangfeng Sun, Haohui Mai, Zhiyao Xie
arXiv AI
Sep 12

Autonomy, Social Norms, and Alignment: Towards a Developmental Framework for Autonomous Artificial Agents

The paper proposes a developmental framework for autonomous artificial agents that emphasizes learning social norms and alignment through direct interaction with dynamic environments. It argues that intrinsic motivations such as curiosity and competence can guide exploration, but also complicate alignment with human goals. By drawing parallels to child development, the authors suggest that regulatory sandboxes serve as pedagogical spaces where agents gradually acquire moral agency and adapt their behaviors through experience and cooperation.

By Marica Notte, Ludovica Marinucci, Vieri Giuliano Santucci
arXiv AI
Sep 12

COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization

COBRA‑Skills is a new framework that treats skill optimization for large language model agents as a budgeted sequential problem over a dynamically evolving candidate set. It uses contextual‑bandit prioritization to focus evaluations on promising or informative candidates and refines the skill population based on execution feedback. In experiments across six agent benchmarks and three target models, COBRA‑Skills outperforms existing methods, cuts optimization cost by 55–58 % compared to SkillOpt, and requires only 50 unique optimization examples per benchmark.

By Pingchen Lu, Xiangyi Wang, Xiang Li, Jie Mao, Zikun Qu, Junfeng Luo, Yao Shu, Bryan Kian Hsiang Low, Zhongxiang Dai
arXiv AI
Sep 12

DeFiFusion: Combining Transaction Events with Smart Contracts to Detect Price Manipulation Attacks

DeFiFusion is a dual‑modal framework that detects price manipulation attacks in decentralized finance by jointly analyzing transaction events and smart contract semantics. It encodes fine‑grained temporal and economic features from transactions and extracts contract logic using large language models, then fuses these signals with a Dual‑Modal Projection‑Fusion Transformer. The approach achieves state‑of‑the‑art performance, recalling 222 of 225 known attacks with 96.10% precision.

By Rui Cao, Shaojing Fan, Liming Fang, Yuchan Liu, Yingying Jiao, Zhenguang Liu
arXiv AI
Sep 12

TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas

TRUST‑SQL tackles Text‑to‑SQL parsing in environments where the full database schema is unknown, a common scenario in enterprise settings with many noisy tables. The method models the problem as a Partially Observable Markov Decision Process and uses a four‑phase protocol to verify only relevant metadata, guided by a Dual‑Track GRPO strategy that separates exploration from execution rewards. Experiments on five benchmarks show significant gains, with the 4B and 8B variants outperforming base models by 30.6% and 16.6% respectively, while matching or exceeding baselines that rely on pre‑loaded schemas.

By Ai Jian, Xiaoyun Zhang, Eryu Guo, Wanrou Du, Jingqing Ruan, Jiangbo Pei, Weipeng Zhang, Ke Zeng, Xunliang Cai
arXiv AI
Sep 12

Causal Past Logic for Runtime Verification of Distributed LLM Agent Workflows

The paper presents a runtime monitoring approach for distributed large language model (LLM) agent workflows. It introduces Causal Past Logic (CPL), an extension of PT‑DTL, allowing guards in if‑constructs and while loops to access the latest causally visible events and variables from other lifelines. The authors adapt the knowledge‑vector monitor to the ZipperGen framework and prove that the locally computed monitor value matches the denotational semantics of the guard at each event.

By Benedikt Bollig
arXiv AI
Sep 12

Generating a Consistent Enterprise: Synthesis and Reference-Free Evaluation of Multi-System Business Data

The paper introduces a synthetic data generator that creates fully consistent, fictional enterprises—complete with workforce, customers, sales, support, and communication records—without relying on any real dataset. It validates realism through a five‑axis scorecard, an adversarial detector, and soundness checks, achieving a mean realism score of 99.1 across 23 generated companies. A second generator produces relational databases from business questions, ensuring qualifying rows and exact labels, and is available as a hosted service and containerized simulators.

By Benjamin Gruenbaum, Doron Porat, Assaf Natanzon, Roy Zavida, Chen Dinachi, Or Itzahary, Omer Niv
arXiv AI
Sep 12

No-Box Vulnerability Analysis: Description-only Detection of Indirect Prompt Injection Vulnerabilities in MCP Servers

The paper introduces no‑box vulnerability analysis, a method that detects security flaws without system access or runtime interaction by examining only the functionality metadata of a target. Using this approach, the authors built MCPSEC to audit Model Context Protocol servers for indirect prompt injection vulnerabilities, evaluating it on 20 servers with 177 tools. MCPSEC identified 143 vulnerable tools, achieving 98.9% recall of verified vulnerabilities, outperforming an LLM baseline.

By Zehua Zhang, Jie Hu, Pratham Hegde, Aditya Maheshbhai Gabani, Souradip Nath, Yibo Liu, Siyu Liu, Hongkai Chen, Hulin Wang, Zhuoer Lyu, Chang Zhu, Divij Handa, Yan Shoshitaishvili, Tiffany Bao, Ruoyu Wang, Adam Doupe
arXiv AI
Sep 12

ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs

ReactHuman is a physics‑grounded benchmark that tests whether multimodal large language models (MLLMs) can make immediate, safety‑critical decisions in simulated humanoid scenarios involving sudden household hazards. The benchmark includes 17 event families, over 1,000 reproducible scenes generated from 240 Hz rigid‑body simulation, and a five‑metric suite evaluating reactions on reasonableness, safety, and physical grounding. Evaluation of seven MLLMs reveals that reactive safety remains unsolved, with models frequently mishandling hazards, relying on appearance over motion, and missing key interception points.

By Yizhan Li, Jianxin You, Mengyang Xiong, Yinhuan Chen, Zicheng Zhao, Dekun Wu, Dongqing Zhang, Bang Liu