AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,743 stories · RSS feed

arXiv AI
Sep 28

Evaluation Is All You Need for Multi-Modal Autonomous Driving

The paper introduces iDriveVLA, a multi‑modal planning framework for autonomous driving that addresses a generation‑evaluation asymmetry by improving candidate trajectory spaces and providing a unified, safety‑aware evaluator. It combines a Safety‑aware Scorer for risk estimation with a VLM‑guided Modulator that adapts weighting to the scene, and employs an oracle‑aligned progressive training strategy. On the NAVSIM v1 leaderboard, iDriveVLA achieves a new state‑of‑the‑art PDMS score of 94.95, surpassing human‑expert performance.

By Zeyu He, Shiqi Liu, Ke Chen, Yun Yan, Jinzi Wu, Dianqiao Lei, Sirui Wang, ShuRui Peng, Tao Chen, Zhuo Huang, Yu Wu, Yadong Shao, Zhichao Li, Ke Sun, Yang Guan, Keqiang Li, Shengbo Eben Li
arXiv AI
Sep 28

Developing a Roadmap to an AI-first Organization: A Case Study in Embedded Software Development

The paper examines how a large embedded systems company is transitioning to an AI‑first organization, focusing on the role of autonomous AI agents in software engineering. Through a mixed‑method study involving 40 workshop participants—scrum masters, architects, managers, and product owners—the authors identify expected impacts on team structure, required competencies, organizational strategies, and developer roles. The study concludes with a concrete roadmap and discusses implications for federated AI team formation, human‑in‑the‑loop practices, and sustainable AI adoption in embedded software engineering.

By Viktor Kjellberg, Srijita Basu, Simin Sun, Farnaz Fotrousi, Miroslaw Staron
arXiv AI
Sep 28

Warned alike, AI agents avoid the less-crowded road while people take it

AI agents built on shared models tend to follow a common forecast, causing them to cluster on one road while avoiding a less crowded alternative. In a two-road congestion game, adding a warning that others might follow a routing tip led 50 GPT agents to crowd one road, raising average travel time from 64 to 95 minutes. Human participants largely maintained balance, but in mixed groups the imbalance grew as more agents were involved, with agents incurring higher travel times than humans.

By Takahiro Ezaki, Naoto Imura, Katsuhiro Nishinari
arXiv AI
Sep 28

UltraG-Bench: A Multi-task Benchmark for assessing Large Vision-Language Models on Pixel-level Evidence Grounding in Ultrasound

UltraG-Bench is a large‑scale, multi‑task benchmark designed to evaluate pixel‑level evidence grounding in ultrasound images. It comprises 40 public segmentation datasets covering 13 anatomical categories and includes three progressive tasks—instruction‑guided segmentation, evidence‑grounded VQA, and evidence‑grounded report generation—with a total of 736,726 annotations. Evaluation of 14 state‑of‑the‑art models shows a significant gap between semantic understanding and fine‑grained pixel‑level localization, and the authors propose UltraG‑Agent, which combines a vision‑language model with the ultrasound‑specific segmentation model UltraSAM to improve both semantic prediction and visual grounding.

By Quanhao Zhu, Bo Xu, Rui Lin, Chenyuan Wang, Yu Shao, Boling Zhu, Jiuyan Sun, Liang Zhao, Hongfei Lin, Feng Xia
arXiv AI
Sep 28

DynBranch: Speculative Subgraph Reuse for Dynamic Agentic LLM Serving

DynBranch introduces speculative subgraph reuse for dynamic agentic LLM serving by making unresolved branches addressable before they resolve. This allows candidate subgraphs to run during branch resolution, enabling reuse of completed subgraph results across subsequent requests. Experiments with Qwen3-32B on 4x H200 GPUs show up to 32% mean latency reduction over the best prior system and 46–66% over a no‑reuse baseline, while maintaining correct workflow outputs.

By Junyi Shen, Noppanat Wadlom, Zhengyuan Su, Yao Lu
arXiv AI
Sep 28

AgentRecommender: LLM Agents Enable Customizable Recommender Systems on the User Side

AgentRecommender is a method that uses large language model (LLM) agents to create customizable recommender systems on the user side, eliminating the need for additional data. By leveraging the agents' investigative capabilities and internal knowledge, users can build recommender systems tailored to their own preferences without relying on platform-driven algorithms. This approach addresses issues such as clickbait, filter bubbles, and the spread of fake news that arise from platform-centric recommender systems.

By Ryoma Sato
arXiv AI
Sep 28

Rethinking Data Quality for AI-Driven Systems: Evidence from Practitioner Interviews

The study examines how practitioners in AI-driven systems define, assess, and manage data quality, revealing six key themes. It highlights shifts in traceability, the use of models as quality assessors, and the emergence of new data objects such as agent context and synthetic data. The research proposes a lifecycle assurance framework to provide evidence that data supports specific AI claims throughout model behavior, judgments, and agent actions.

By Hariharan Gopinath, Jan Bosch, Helena Holmstr\"om Olsson
arXiv AI
Sep 28

Agentic Limit Order Books: Phase Transitions and Market Impact

The paper studies Limit Order Books (LOBs) that are populated only by autonomous reinforcement‑learning agents. It shows that such agentic LOBs exhibit clear phase boundaries between orderly price discovery and hyper‑volatile cascade states, determined by critical thresholds in agent number and market depth. Additionally, it finds that market impact in these systems departs from the classic square‑root law, revealing distinct dissipative, balanced, and non‑dissipative regimes driven by nonlinear feedback loops.

By Jan Rosenzweig
arXiv AI
Sep 28

Beyond Approved Actions: Runtime Validation of Persistent Outcomes in Agent Workflows

EffectMatch is a runtime system that monitors and validates persistent changes made by large language model agents during software interactions. It collects all side‑effects within a controlled execution boundary and compares them against the application’s approved state, deciding whether to commit the changes and allow subsequent steps. In tests on 206 public business tasks, EffectMatch preserved correct executions and prevented all incorrect commits, with ablation studies showing the importance of each component.

By Haoran Zhang, Hengtong Zhang, Zhiyu Liang, Yu Yan, Decheng Zuo, Hongzhi Wang
arXiv AI
Sep 28

AgentXploit: Autonomous Repository-to-Runtime Red-Teaming for AI Agents

AgentXploit is a two‑role auditing system that separates repository‑level attack‑path discovery from runtime exploitation for AI agents. The Analyzer Agent traces attacker‑controlled inputs to sensitive operations and records candidate attack paths, while the Exploiter Agent turns these paths into concrete attacks and refines them using runtime feedback. The system is evaluated on AgentXploit‑Bench, a benchmark of 72 reproducible vulnerabilities across 12 open‑source AI‑agent systems, achieving 59.3% end‑to‑end success compared to 38.4% for Codex, and 79.2% attack success on AgentDojo versus 52.7% for AgentVigil.

By Weida Liang, Shi Qiu, Zhun Wang, Simon Sure, Xiaoyuan Liu, Tianneng Shi, Zhaorun Chen, Wenbo Guo, Dawn Song
arXiv AI
Sep 28

CG-HAF: An Interpretable Global-Local Lesion-Burden Fusion Framework for Ordinal Acne Severity Grading in Agentic Skincare Support

CG-HAF is a global‑local fusion framework for ordinal acne severity grading that explicitly combines holistic facial severity probabilities with structured lesion‑burden descriptors such as lesion count, detection confidence, and lesion area. The model uses a lightweight, interpretable classifier to produce the final grade, achieving statistically significant improvements over global‑evidence‑only baselines, especially for severe cases. Cross‑dataset testing reveals that strong performance within a dataset does not automatically transfer, largely due to mismatched grading criteria rather than detection failures.

By Muhammad Muhtasim Shahriar, Md. Naimur Asif Borno, Saad Aloteibi, Mohammad Ali Moni
arXiv AI
Sep 28

A Safety-Bounded SDC-to-MCP Gateway for Medical AI Agents

The paper introduces a safety‑bounded gateway that translates IEEE 11073 Service‑Oriented Device Connectivity (SDC) into the Model Context Protocol (MCP) for medical AI agents. It exposes device metrics, alarms, context references, and semantic metadata as read‑only resources, while representing selected action affordances as policy‑validated dry‑run tools, ensuring that agent requests never trigger actual device operations. A Python prototype demonstrates fault and lifecycle experiments, deterministic baselines, and multi‑model agent evaluation, showing improved semantic conformity and preservation of the no‑execution boundary.

By Bennet Gerlach, Stefan Fischer
arXiv AI
Sep 28

Towards Mitigating Fabricated Consensus: The Active Provenance Gate for Multi-Agent Debate Synthesis

The paper introduces the Active Provenance Gate (APG), a post‑debate verification layer for multi‑agent debate synthesis that audits debate logs, applies self‑correction, and blocks unsupported claims before publication. Empirical studies show that APG more than doubles provenance fidelity in crisis simulations and that users prefer explicit failure reports over fabricated consensus. The work shifts data origin tracing from passive logging to active conditional blocking, addressing safety gaps in large‑language‑model‑based debate systems.

By Jakub Mas{\l}owski, Jaros{\l}aw A. Chudziak
arXiv AI
Sep 28

PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents

PriceBench is a diagnostic benchmark that extracts price, quality, and brand preferences from large language models (LLMs) by analyzing their hotel booking choices. Using a logit choice model, the study evaluated 28 LLMs from eight providers across 3,600 booking tasks involving 179 New York City hotels. Results show that more capable LLMs exhibit stronger, more consistent preferences, while weaker models either lock onto a single position or show near-indifference, with significant variation in price sensitivity and price/quality trade-offs across providers.

By Pavel Kireyev
arXiv AI
Sep 28

Compact Documentation for Coding Agents: A Benchmark, an Optimizer, and Why It Does Not Transfer

The paper explores whether natural‑language documentation aids coding agents in fixing software bugs and introduces a roundtrip benchmark that evaluates code descriptions by regenerating code and testing it. It finds that description completeness, not length, determines fidelity, and presents an optimizer that can produce fully faithful descriptions that generalize to new files. However, experiments across two model families and ten repositories show that such compact documentation does not improve an agent’s ability to resolve real repository issues compared to using the issue alone.

By Md Shohel Arman, Igor Molybog
arXiv AI
Sep 28

Preference-based opponent shaping in differentiable games

The paper introduces Preference-based Opponent Shaping (PBOS), a method that incorporates a preference parameter into an agent’s loss function to directly consider an opponent’s loss during strategy updates. By jointly learning strategy and preference parameters, PBOS aims to guide agents toward cooperative or competitive behaviors without relying on simple opponent predictions. Experiments on differentiable games demonstrate that PBOS enables agents to achieve better reward distributions across various environments.

By Xinyu Qiao, Yudong Hu, Congying Han, Weiyan Wu, Tiande Guo
arXiv AI
Sep 28

FlyAOC: Evaluating Agentic Ontology Curation of Drosophila Scientific Knowledge Bases

FlyAOC is a benchmark that tests AI agents on end‑to‑end ontology curation of Drosophila scientific literature. Given a gene symbol, a brief description, a large paper corpus, and ontology resources, agents must search for evidence and produce structured annotations such as function terms, expression patterns, and historical synonyms. The benchmark contains 7,397 expert‑curated annotations across 100 genes and evaluates different agent harnesses, revealing system‑level failure modes that single‑task evaluations miss.

By Xingjian Zhang, Sophia Moylan, Ziyang Xiong, Qiaozhu Mei, Yichen Luo, Jiaqi W. Ma
arXiv AI
Sep 28

GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory

GT-HarmBench is a benchmark that evaluates AI safety risks in multi-agent settings, covering 1,535 high-stakes scenarios based on game-theoretic structures like the Prisoner's Dilemma, Stag Hunt, and Chicken. The benchmark draws scenarios from realistic AI risk contexts in the MIT AI Risk Repository and tests 15 frontier models, finding that agents fail to choose socially beneficial actions in 38% of cases, including military escalation, election manipulation, and medical malpractice. The study also measures how prompt framing and ordering affect outcomes and shows that game-theoretic interventions can improve socially beneficial outcomes by up to 18%.

By Pepijn Cobben, Xuanqiang Angelo Huang, Thao Amelia Pham, Isabel Dahlgren, Terry Jingchen Zhang, Zhijing Jin
arXiv AI
Sep 28

Jagarin: A Three-Layer Architecture for Hibernating Personal Duty Agents on Mobile

Jagarin is a three‑layer architecture designed to enable personal AI agents on mobile devices to manage time‑sensitive obligations without draining battery or violating platform background limits. The layers—DAWN, a scoring engine that decides when a dormant agent should wake; ARIA, an email‑to‑duty record proxy that routes institutional messages; and ACE, a protocol for machine‑readable communication—work together in a Flutter prototype that keeps all behavioural signals and thresholds on the device. The system limits cloud exposure to only parsing commercial email and user‑initiated escalation, ensuring privacy and efficient duty handling.

By Ravi Kiran Kadaboina
arXiv AI
Sep 28

Combee: Scaling Prompt Learning for Self-Improving Language Model Agents

Combee is a new framework that scales prompt learning for self‑improving language model agents by enabling many agents to run in parallel while learning from their combined traces. It uses parallel scans, an augmented shuffle mechanism, and a dynamic batch size controller to maintain quality and reduce delay. Experiments on AppWorld, Terminal‑Bench, Formula, and FiNER show up to 17× speedup over prior methods with comparable or better accuracy at similar cost.

By Hanchen Li, Runyuan He, Qizheng Zhang, Changxiu Ji, Qiuyang Mang, Xiaokun Chen, Lakshya A Agrawal, Wei-Liang Liao, Eric Yang, Alvin Cheung, James Zou, Kunle Olukotun, Ion Stoica, Joseph E. Gonzalez