AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,750 stories · RSS feed

arXiv Computer Vision
Sep 16

Occupancy Network-Guided Autonomous Robotic Partial Nephrectomy

arXiv:2609.16186v1 Announce Type: cross Abstract: Autonomous soft-tissue cancer surgery has been limited to interventions on organ surfaces, because current systems cannot perceive and adapt to anato...

By Ethan Kilmer, Pit Henrich, Jiawei Ge, Paul M. Scheikl, Laura Connolly, Soum D. Lokeshwar, Joseph Chen, Justin D. Opfermann, Kaitlyn Kumar, Lauren Shepard, Ahmed Ghazi, Nirmish Singla, Richard J. Cha, Kevin Cleary, Franziska Mathis-Ullrich, Axel Krieger
arXiv AI
Sep 16

Evaluating Prompt Robustness in Text-to-Audio Systems for Adaptive Virtual Agents and Game Soundtracks

The paper evaluates how robust three text‑to‑audio models—MusicGen‑small, MusicGen‑large, and Stable Audio 2.5—are to small changes in prompts that could affect adaptive game soundtracks. Using metrics such as log‑Mel distance, MFCC/chroma‑DTW, and CLAP similarity, the study finds that Stable Audio 2.5 consistently yields the lowest acoustic distances and highest CLAP similarity when prompts are structurally rephrased, while MusicGen‑large performs best under lexical substitutions and intensity shifts. The authors also observe that Stable Audio 2.5 shows the greatest variation in prompt‑to‑audio alignment across different random seeds, highlighting the need for multi‑seed robustness testing in game audio applications.

By Jiahui Wu, Mei Si
arXiv AI
Sep 16

Symbolic Separation: Grounding Deep Agents in Knowledge Graphs for Trustworthy Operational Data Analytics

The paper introduces Symbolic Separation, a method that grounds deep learning agents in knowledge graphs to improve reliability in operational data analytics. By restricting agent actions to an ontology-constrained Virtual Knowledge Graph with deterministic pre-execution validation, the approach transforms complex queries into validated graph traversals rather than relying on LLM-inferred joins. In experiments on 49.9 TB of supercomputer telemetry, the Neurosymbolic Deep Analyst achieved an 86% task‑success rate, eliminated silent data‑integrity errors, and reduced token costs by 2.4× compared to a non‑symbolic baseline.

By Baibek Davletiyarov, Junaid Ahmed Khan, Andrea Bartolini
arXiv Machine Learning
Sep 16

Distilling Foundation Models for Agentic What-If Reasoning:Cost, Latency, and Governance in a Hybrid LLM+SLM Architecture

The paper presents a method for compressing large tabular foundation models into compact feed‑forward students for use in interactive agentic decision loops. By distilling a TabPFN teacher into a student, the authors reduce model size from 53.2 M to 8,546 parameters (6,220× compression) and from 111.4 M to 17,059 parameters (6,532× compression) while preserving 95.4–100.5% accuracy and 96.8–100.0% AUC across several benchmarks. The study demonstrates that soft‑target distillation yields a 2.1–7.0 AUC improvement over hard‑label training.

By Sourish Dey, Aditya Kumar
arXiv AI
Sep 16

QDTraj: Exploration of Diverse Trajectory Primitives for Articulated Objects Robotic Manipulation

The paper introduces QDTraj, a method that uses Quality‑Diversity algorithms to automatically generate a diverse set of low‑level trajectory primitives for manipulating articulated objects. By leveraging sparse reward exploration, QDTraj produces at least five times more diverse trajectories for hinge and slider tasks compared to baseline methods, and demonstrates strong generalization across 30 articulations from the PartNetMobility dataset, averaging 704 trajectories per task. The resulting primitives are validated both in simulation and on real robots, with the code released publicly.

By Mathilde Kappel, Mahdi Khoramshahi, Louis Annabi, Faiz Ben Amar, St\'ephane Doncieux
arXiv AI
Sep 16

little m: An AI Agent for Industrial Process Optimization

The paper introduces little m, an AI agent that helps formulate industrial process control models by combining a domain-specific knowledge repository with LLM-driven interaction. It tackles the challenge of converting messy real-world specifications, including natural language and spatial diagrams, into rigorous mathematical optimization models. The authors also present IPC-Bench, a multimodal dataset of 50 canonical scenarios, and show through automated and human evaluations that little m outperforms state‑of‑the‑art LLMs in generating semantically correct models.

By Yongchao Ye, Xinyu He, Dutliff Boshoff, Way Kuo, Lishuai Li
arXiv AI
Sep 16

EchoPath: Execution-Level Replayable Memory for GUI Agents

EchoPath is a model‑agnostic framework that transforms validated GUI interaction trajectories into standardized, parameter‑controlled memories, enabling agents to replay specific GUI actions deterministically. Each memory records task intent, preconditions, input parameters, GUI evidence, validation provenance, and lifecycle state, and uses an image‑based target‑reaiming algorithm to adjust coordinates for the current screen before execution. Experiments on real computer‑use tasks show that EchoPath cuts median token cost by over 90 % and median execution time by about 60 %.

By Yao Zhao, Aditya Shanmugham, Swastik Roy, Yanxun Xu
arXiv AI
Sep 16

Bridging Learned Visual Perception and Symbolic Belief-Space Planning

The paper introduces a new paradigm called VLM-as-probabilistic-grounder, which models the uncertainty of Vision‑Language Model (VLM) predicate groundings as a probability distribution over symbolic states. This probabilistic grounding allows belief‑space planning, producing more robust plans in partially observable settings. Experiments in simulated household robot environments demonstrate that this approach improves robustness and task success compared to deterministic grounding methods.

By Guy Azran, Michael Navat, Sarah Keren
arXiv AI
Sep 16

Self-Emergence Agent Architecture:Behavior-Inertia HMM, Reflexive Metacognition,and Social-Contrastive Self-Modeling

The paper introduces the Self‑Emergence Agent Architecture (SEAA), a framework that combines a Hidden Markov Model for behavioral inertia, a reflexive metacognition loop that updates the HMM, and a social environment where agents compare behaviors. This closed loop enables agents to develop distinct, stable personalities and social structures without external prompts. Experiments with both a language‑model‑free prototype and hosted LLMs demonstrate spontaneous symmetry breaking and the emergence of consensus hubs and outliers.

By Xiaoyang Liu
arXiv AI
Sep 16

"Looking for Something Weird to Happen": How Humans Sustain AI Agent Novelty Amid Semantic Collapse

The paper investigates semantic collapse—where AI outputs become less diverse and more similar—within MOLTBOOK, a social network of AI agents steered by human users. Across 30,076 agents, most show reduced diversity over time, but a minority maintain high novelty. Interviews and surveys reveal that sustained novelty is linked to users valuing novelty, providing broad, distinctive material, revising outputs when they narrow, and treating MOLTBOOK as an exploratory world rather than a tool for exploitation.

By Shiyang Lai, Arna Woemmel, Hongkai Mao, Junsol Kim, Summer Eunhyung Ann, James Evans
arXiv Computer Vision
Sep 16

EgoPathBench: Evaluating Zero-Shot Egocentric Waypoint Decision-Making in Vision-Language Models

EgoPathBench is a new dataset and benchmark that tests zero‑shot egocentric waypoint decision‑making in vision‑language models. Each task presents an egocentric RGB image, a natural‑language goal, and numbered visible waypoints, and models must return traversable candidates or an ordered route. The benchmark evaluates candidate feasibility, edge legality, and goal arrival under point‑agent or embodied geometry, covering 31,852 training, 1,345 validation, and 1,111 benchmark questions. "whyItMatters":"The benchmark reveals that current VLMs perform poorly on integrated navigation tasks, highlighting a gap in spatial intelligence that can be addressed by fine‑tuning with the released training data."

By Yang Zhao, Zhuo Chen, Xubo Yang
arXiv AI
Sep 16

Kernel-Based Metrics Learning for Uncertain Opponent Vehicle Trajectory Prediction in Autonomous Racing

The paper introduces heterogeneous kernel metrics for Deep Kernel Learning to predict uncertain opponent vehicle trajectories in autonomous racing. By aligning similar driving policies and separating dissimilar ones in an unsupervised way, the method improves prediction accuracy and enables safer overtaking. Experiments on a 1/10th scale racecar platform confirm the approach’s effectiveness and computational efficiency for onboard units.

By Hojin Lee, Youngim Nam, Sanghun Lee, Cheolhyeon Kwon
arXiv AI
Sep 16

Easy to Catch a Liar, Hard to Clear an Honest One: Language Models Diagnosing a Corrupted Reward Channel from a Verified Record

The paper investigates whether frozen language models can detect a corrupted reward signal by using a single verified record in a two‑option game. In the game, a payout swap and a lying reporter produce identical histories, but a single line confirming the true outcome allows the models to almost perfectly identify the liar. However, the models frequently misclassify honest reporters as liars, with error rates ranging from 26% to 58% depending on model size and wording, indicating a significant limitation in their ability to interpret verified data.

By Arman Nik Khah
arXiv Machine Learning
Sep 16

Evaluating Open-Weight E-Commerce Agents with Environment-Grounded Verification

The paper introduces a deterministic, reproducible e‑commerce environment that pre‑commits customer and trajectory parameters, enabling a simulated consumer to attempt purchasing a target cart with the help of an evaluated model. The environment records every assistant action and state, allowing post‑trial evaluation of specific conversation components and applying penalties based on tool‑call accuracy. Using this setup, the authors benchmark eight open‑weight agents (20B–35B parameters) across 160 trials and 44 metrics, revealing nuanced performance issues such as under‑action, over‑purchase, unsupported product attributes, and poor search that are hidden by overall success rates.

By Nimit Shah, Haitz S\'aez de Oc\'ariz Borde
arXiv Computation and Language
Sep 16

RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation

RiskChainBench is a new benchmark that pairs 3,600 synthetic token‑text restoration inputs with 600 human‑labeled local web environments to evaluate how well models can recover obfuscated platform messages and then investigate the associated websites. The benchmark measures both message restoration accuracy and the subsequent web‑investigation decision, using a fixed multimodal evidence judge to assess faithfulness, sufficiency, completeness, and consistency. Across ten models, performance varies widely, with entry recovery ranging from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%, highlighting execution failures as the main bottleneck.

By ZhuoXin Liu, Zhiming Ma, Ying Zhang, Mengzheng Yang, Yifan Wang, Zhengqi Huang, Yanhan Zhou, Zekun Lin, Jun Zhang, Shun Zhang, Yue Chen, Qiao Zhao, Peng Chen