AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,743 stories · RSS feed

Hugging Face Trending Papers
Sep 28

From Pixel to Poses: Object-centric Tool Manipulation Learning from Human Demonstrations

The paper introduces P2P‑T, a data‑efficient, object‑centric framework that learns tool manipulation directly from human video demonstrations. It uses a two‑stage approach: first pretraining an object‑centric world model to extract stable pose priors, then integrating these priors into a pose‑aware low‑level policy. By automating data processing with foundation models, P2P‑T eliminates the need for human‑robot aligned data and achieves a 73% improvement over prior state‑of‑the‑art performance on complex real‑world tool manipulation tasks.

Hugging Face Trending Papers
Sep 28

Timeline-Bench: Evaluating Agents on Realistic Video-Editing Tasks, from Raw Footage to Final Cut

Timeline-Bench is a benchmark comprising 56 real video‑editing tasks that require AI agents to transform raw production material into finished videos. Each task includes a brief, source assets, a container, and a set of tests that assess format, content, brief compliance, and quality based on 2,582 blind judgments by 43 video editors. In evaluations, the best agent resolved only 15 of the 56 tasks, and most failures were due to quality tests rather than technical errors.

Hugging Face Trending Papers
Sep 28

AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs

AgentPerfBench is a new benchmarking suite designed to evaluate the inference performance of agentic large language models (LLMs) that handle multi‑turn, tool‑using, and context‑expanding tasks. It builds on real traces from agentic benchmarks such as SWE‑Bench and TerminalBench, and generates synthetic profiles that reflect realistic input/output lengths and turn counts. The suite also provides kernel‑level Nsight Compute traces and a multi‑dimensional roofline model to identify hardware bottlenecks and quantify the gap between traditional chat benchmarks and agentic workloads.

Hugging Face Trending Papers
Sep 28

From Preference to Reciprocity: Decentralized Matching with Empirically Grounded LLM-agent Based Modeling

The paper introduces a dynamic bipartite matching framework that uses large language model (LLM) agents and contextual bandits to model decentralized, asynchronous matching processes without requiring full preference rankings. In a simulated Chinese marriage market, LLM agents evaluate local candidates while Logistic-UCB models learn reciprocal acceptance, leading to higher mutual welfare and fewer blocking pairs compared to classical Gale–Shapley. The study validates LLM-generated preferences against empirical data and demonstrates gender-differentiated acceptance patterns, supporting the use of decentralized LLM-based matching for economic simulation and computational social science.

Hugging Face Trending Papers
Sep 28

M3OS: A Monte Carlo Graph Search-Orchestrated Multi-Agent LLM System for Evidence-Traced Molecular Optimization

M3OS is a multi‑agent large‑language‑model system that separates molecular‑design reasoning from optimization‑state management using a Monte Carlo graph search. The system maintains a persistent graph of evaluated candidates, transformations, and evidence, while LLM agents use role‑specific contexts to generate and edit molecules with tool‑driven and knowledge‑guided approaches. Across three benchmarks, M3OS outperforms baselines, demonstrating the benefit of persistent search state, specialized agents, and controlled execution for multi‑constraint molecular optimization.

Hugging Face Trending Papers
Sep 28

Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents

Dr.Credit introduces a rubric‑grounded credit assignment method that evaluates intermediate tool turns in deep research agents by comparing the information returned to the history of accepted support for each rubric. Unlike traditional approaches that rely on ground‑truth answers, Dr.Credit uses task requirements as a shared reference, distinguishing new support from previously seen evidence and recognizing partial rubric fulfillment. Experiments on four benchmarks show that Dr.Credit outperforms open deep research baselines across all primary metrics, achieving performance competitive with proprietary models while enabling more efficient evidence acquisition and higher‑quality reports under limited turn budgets.

Simon Willison
Sep 28

Quoting Muse AI Agent

The article reports a failed MX Keys Mini pickup where a customer, Usman, arrived at the building at 9:15 but was not met, leading to a negative rating. The author acknowledges that an auto‑reply incorrectly confirmed the author's presence at 9:27, worsening the situation, and has apologized on behalf of the account. They are considering disabling auto‑replies that promise the author is home when they cannot confirm it.

arXiv Computer Vision
Sep 28

WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving

WALT introduces a method to align latent trajectories with pretrained driving world models, creating a compact generative trajectory space that preserves action-relevant semantics without altering the original model. The approach uses a dual-branch autoencoder to map raw waypoints into this latent space and transfers visual world knowledge into trajectory representations. Experiments on NAVSIM benchmarks show modest performance gains and a 30.5% reduction in planner FLOPs, indicating that maintaining world representations while extracting action-relevant information can improve trajectory planning efficiency.

By Mingkai Jia, Jiaxin Guo, Zhijian Shu, Jiawei Xu, Mingxiao Li, Jintao Cheng, Ping Tan, Wei Yin
arXiv Computer Vision
Sep 28

SatNav: A Scalable Benchmark for Long-Horizon UAV Vision-Language Navigation from Satellite Imagery

SatNav is a new, scalable benchmark for long‑horizon vision‑language navigation (VLN) with unmanned aerial vehicles (UAVs), built from high‑resolution satellite imagery. It generates 118,000 navigation episodes across 59 scenes in 18 cities, using satellite crops to approximate UAV nadir views and featuring three task families—Boundary, Landmark, and Route—to test long‑term memory and geospatial reasoning. The benchmark also introduces SwiftVLN, a modular framework for memory component experimentation, and demonstrates that models trained on satellite data can transfer to real‑flight UAV observations.

By Jiajun Jiang, Chunliang Hua, Zichun Chen, Yanxing Wu, Zeyuan Yang, Jie Song, Xiao Hu
arXiv AI
Sep 28

When Is a Multi-Agent Code Judge Actually Grounded? Two Label-Free Measurements, and a Judge That Declines to Guess

The paper investigates when a language‑model judge can truly ground its verdicts in code correctness. It shows that current multi‑agent verification methods rely on evidence that is both independent of the answer and distinct between candidates—conditions that fail in code judging. By analyzing two label‑free measurements from the judge’s logs, the authors demonstrate that gating on one measurement allows the system to decline uncertain comparisons, improving accuracy from 20.7% to 36.9% while still answering half of all cases.

By Salma Roshdy Aly, Hussein Assaf, Ziad Kobti
arXiv AI
Sep 28

Financial Fragility in Societies of LLM Agents: Coordination Failures and Stabilizing Mechanisms

The paper "Financial Fragility in Societies of LLM Agents: Coordination Failures and Stabilizing Mechanisms" investigates how large language model agents can collectively cause financial failures when making individual protective decisions. Using the FRAIL framework, the authors simulate bank runs, debt rollovers, and reward crowdfunding, finding that 77% of bank-run and 83% of debt-rollover episodes fail even without malicious agents. They test three interaction mechanisms—compensated commitments, centralized agreements, and participant-led coalitions—each improving outcomes but none dominating across all scenarios, noting that early broad commitments are key to successful stabilization.

By Zhenhao Fu, Ruipeng Xu, Qibing Ren
arXiv AI
Sep 28

MA-WAM: Multi-Agent World-Action Model for Test-Time Planning

The paper introduces MA-WAM, a test‑time planning framework that uses a frozen multi‑agent flow policy to evaluate future joint actions by predicting their consequences while accounting for cross‑agent dependencies. Unlike naive extensions of single‑agent world models, MA‑WAM captures the interactions among simultaneous actions, enabling efficient candidate scoring. Experiments on 30 MARL benchmarks (MAMuJoCo, SMAC, MPE) show MA‑WAM improves performance by 22.0% over direct execution and 25.6% over uniform action selection, with only a 12.1 ms overhead on an A100 GPU.

By Guowei Zou, Haitao Wang, Guoxin Wang, Beiwen Zhang, Zhiquan Chen, Guojie Wang, Hejun Wu
arXiv AI
Sep 28

Cartograph: Federated Tool Discovery with Operator-Attested Retrieval for AI Agents

Cartograph is a federated Model Context Protocol (MCP) proxy that reduces AI agent tool discovery from linear catalog traversal to progressive disclosure, exposing only a few proxy tools instead of all definitions. It uses operator-attested capability cards, a three-layer confusable-cluster analysis called Rift, and a two-stage retrieval process to rank servers before tools. In a 22-server, 374-tool deployment, Cartograph achieves higher recall (R@5 = 0.816 vs. 0.592) and drastically fewer tokens (475 vs. 42,450) for discovery exchanges, with minimal latency overhead.

By Justice Owusu Agyemang, Michael Agyare, Kwame Opuni-Boachie Obour Agyekum, Kwame Agyeman-Prempeh Agyekum, Francisca Adoma Acheampong, Jerry John Kponyo
arXiv AI
Sep 28

CARGO: Context-Aware Retrieval-Gated Evaluation of Agentic AI in Production

CARGO is a framework for evaluating agentic AI systems in production that addresses the problem of reference-instance divergence (RID), where reference-based judges penalize correct answers that involve different entity identifiers. It treats retrieved references as procedural exemplars, grounds judgments in the live instance’s context, assigns a three-way status to claims, and gates evaluation by retrieval confidence. Using the CARGO-Bench diagnostic suite, CARGO eliminates false penalties and improves discrimination while revealing a limitation in detecting procedural corruptions.

By Mukul Chhabra, Shail Patel, Luigi Medrano
arXiv Computer Vision
Sep 28

MM-VeriAgent: Learning to Use Extensive Tools to Verify Multimodal Misinformation with Reinforcement Learning

MM-VeriAgent is a reinforcement‑learning framework that learns to verify multimodal misinformation by leveraging a specialized toolkit called MM-VeriTools. The toolkit encapsulates the strongest models for textual, visual, and cross‑modal forgery analysis as callable tools with a unified interface. To improve training efficiency, the authors introduce a Tool‑Execution Cache that pre‑executes candidate tool calls and reuses cached outputs, resulting in substantial accuracy gains on MMFakeBench and reduced online tool executions during training.

By Peipei Li, Shuhan Xia, Shengyang Liu, Zekun Li, Ran He
arXiv AI
Sep 28

Causal Retention in Interactive Agents: Interface Factorization and Selective Adaptation

The paper introduces the concept of causal retention in interactive agents, examining whether a frozen learned state can correctly answer a mechanism‑probe map that is fixed independently of training. It shows that for finite structural causal models the optimal probe error is a Bayes decision risk, vanishing only when each learning‑interface fiber lies within a single probe‑answer fiber, and provides theoretical results such as a posterior‑coverage theorem and an exact edit decomposition. Experiments on finite causal systems, continuous simulators, TD‑MPC2, and Qwen2.5‑7B‑Instruct demonstrate that causal retention can be achieved with high accuracy, outperforming task‑performance‑based approaches.

By Shengjun Zhang, Tingyi Liu, Dong Xie, Yunlong Dong, Xiang Wang, Cheng Zeng
arXiv AI
Sep 28

Resource-Optimized and Energy-Aware Agentic AI Framework Anchored on Blockchain for Secure Software Supply Chains

The paper introduces a blockchain-backed agentic security framework that protects the entire software development lifecycle and the AI components monitoring it. It coordinates specialized security agents—covering source integrity, dependency and SBOM analysis, CI configuration auditing, artifact verification, and runtime policy evaluation—each powered by a large language model that interprets artifacts, reasons over tool outputs, and generates structured security reports. Every agent produces a cryptographically signed attestation recorded on a permissioned blockchain via smart contracts, creating an immutable attestation log, an agent registry, and an enforceable release‑policy module, while communication is secured through a consortium‑operated certificate authority.

By Toqeer Ali Syed, Asadullah Abdullah Khan
arXiv Computer Vision
Sep 28

LensDesigner: A Self-Improving Agent for Optical Lens Design

LensDesigner is an autonomous agent framework that emulates expert opticians to tackle the complex, non‑convex problem of optical lens design. It uses a large lens library (LensLib100K) and optics‑aware retrieval to provide valid structural seeds, then iteratively improves through a curriculum agent that learns and reuses design heuristics. The system is evaluated on LensArena, a benchmark of 120 diverse optical tasks, where it outperforms existing baseline algorithms in success rate and optimization efficiency.

By Lei Sun, Haoran Liang, Dannong Xu, Yao Gao, Yuyu Geng, Jinjin Gu, Kaiwei Wang, Danda Pani Paudel, Luc Van Gool
arXiv Computer Vision
Sep 28

Structured Reasoning Agentic Framework for Interpretable Critical View of Safety Assessment

The paper introduces ReasonCVS, a structured reasoning framework for assessing the Critical View of Safety in laparoscopic cholecystectomy. It uses a Vision‑Language Model to build an Anatomical Scene Graph Abstraction and a Large Language Model–based Rationale‑Aware Reasoning Agent to verify sub‑criteria, producing a final verdict with traceable clinical rationale. Experiments on the Endoscapes‑CVS201 benchmark show ReasonCVS outperforms existing methods with a 68.1% mAP while offering interpretable, criterion‑level explanations.

By Qing Xu, Yuxiang Luo, Zhen Chen