AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,750 stories · RSS feed

arXiv Computer Vision
Sep 17

In-Context Robot Learning with VLM Agents

arXiv:2609.19138v1 Announce Type: new Abstract: Enabling robots to adapt to unfamiliar environments as readily as humans remains a moonshot goal of embodied AI. No finite collection of demonstrations...

By Dongzhou Cheng, Taoran Yi, Ye Fang, Xingwu Zhang, Fan Feng, Yixuan Li, Gengxiong Zhuang, Rongze Wang, Shuai Yang, Wei Song, Weizhi Xue, Minyan Wu, Jie Gui, Jiaqi Wang, Tong Wu
arXiv AI
Sep 17

Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents

The paper introduces EvoSkill-GUI, a training‑free framework that enables GUI agents to evolve their skills during deployment. Each skill is packaged with metadata, executable plans, and recovery rules, and the system follows a reflect‑revise‑reuse loop where the agent instantly revises skills based on execution feedback. Experiments on MobileWorld, AndroidWorld, and OSWorld show consistent performance gains up to +16.2% without any additional training.

By Bofan Chen, Boxuan Zhang, Fei Tang, Zhengxi Lu, Yong Du, Tongbo Chen, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
arXiv Machine Learning
Sep 17

Composite-Gradient Learning for Shared Control Authority Between Deep Reinforcement Learning and Model Predictive Control

The paper introduces Composite‑Gradient Learning (CGL), a method that explicitly incorporates a model predictive controller (MPC) into the training of a deep reinforcement learning (DRL) agent by treating their control inputs as a joint action. CGL updates the DRL policy while accounting for the interaction with the MPC, unlike prior approaches that view MPC merely as part of the environment. Experiments on two freeway traffic networks show that CGL performs better than alternative methods when the interaction between DRL and MPC is strong, though overall gains are modest.

By Giray \"On\"ur, Azita Dabiri, Bart De Schutter
arXiv AI
Sep 17

Multitask Reinforcement Learning for Assisting Choice Model Specification

Delphos is a multitask reinforcement learning framework that automates discrete choice model specification by treating it as a sequential decision-making problem. It learns transferable specification strategies across multiple transport choice datasets using a DeepSet‑Q architecture, enabling a shared policy to adapt to varying variable sets. Trained on nine datasets, Delphos outperforms single‑task agents, and when applied to unseen Swissmetro and Decisions datasets, it quickly identifies competitive specifications with higher log‑likelihoods than existing methods.

By Gabriel Nova, Stephane Hess, Sander Van Cranenburgh
arXiv AI
Sep 17

Social Laws for Multi-agent Coordination in Stochastic Environments

The paper extends the concept of social laws from deterministic, goal-based multi‑agent systems to stochastic, reward‑based environments. It introduces a formalism for defining and verifying the robustness of these laws, including a new metric called α‑robustness that quantifies the utility each agent can guarantee while following the law. The authors present a verification approach that reduces the problem to solving multiple Markov decision processes and demonstrate the framework’s potential through empirical evaluations on toy environments.

By Rolando Fernandez, Caleb Probine, Tyler Lee, Jeffrey Chen, Erez Karpas, Muhammad Arrasy Rahman, Peter Stone, Ufuk Topcu
arXiv AI
Sep 17

StableEval Arena: A Cost-Aware Agentic Benchmark for Stablecoin Price Stability Prediction

StableEval Arena is a cost‑aware benchmark framework designed to evaluate agentic AI systems on stablecoin peg‑risk prediction. It tests LLM‑backed agents by diagnosing peg stress and forecasting deviations from the one‑dollar peg over a hidden seven‑day horizon, using leakage‑safe historical replay with exchange price‑volume data and market‑context features. The benchmark includes a 120‑case stress‑enriched validation block and a 507‑case natural‑distribution full‑arena evaluation, measuring prediction quality, calibrated‑label behavior, structured‑output reliability, latency, token consumption, and estimated inference cost across six LLM‑backed agent configurations and baselines.

By Sean Wan, Dongping Liu, Luyao Zhang
arXiv AI
Sep 17

PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?

The paper introduces PACT, a benchmark designed to evaluate how well enterprise AI assistants follow compliance rules when faced with various pressures such as persistent users or hurried managers. PACT covers twelve regulated domains and forty-eight realistic multi‑turn scenarios, pairing each rule with a shortcut that violates it and applying different pressures across wording and system‑prompt modes. Using PACT, the authors profile six metrics of compliance and aggregate them into a PACTScore, revealing significant variability among 22 LLM models and that even top performers misapply rules 6–10% of the time, with user pressure increasing violations by 65% on average.

By Mika Okamoto, Ansel Kaplan Erol
arXiv AI
Sep 17

Ask the Tool, Don't Guess: Agent Tool Calls Hold Their Progress, and the Serving System Should Read It

The paper argues that agentic systems waste time and memory by guessing how long tool calls will take, rather than using explicit progress signals from the tools themselves. It demonstrates that tools can report their remaining work or imminent completion, and that incorporating this feedback into serving systems dramatically improves cache decisions and reduces token latency. The authors show that this approach outperforms existing predictors and works robustly across different environments.

By Yipeng Liu, Yingqiang Zhang, Feifei Li, Huanchen Zhang
arXiv AI
Sep 17

Taming the Agentic RAN: Stability-Guaranteed Arbitration of Autonomous AI Agents in O-RAN

The paper demonstrates that autonomous AI agents (rApps) in an O-RAN control plane can independently close control loops over shared radio resources, leading to unsafe interactions when multiple agents pursue different objectives. The authors introduce AURA, a lightweight arbitration layer that enforces feasibility invariants, dwell times, and a deadband to ensure stable operation. Implemented on an OpenAirInterface testbed, AURA reduces shared-state excursions by more than an order of magnitude and virtually eliminates cross-slice throughput starvation while maintaining latency compliance for the protected slice.

By Seyed Bagher Hashemi Natanzi, Bo Tang
arXiv AI
Sep 17

MIRAGE: How Conversation State Shapes Historical Evidence Use in Multimodal Personal Agents

MIRAGE is a controlled study that examines how multimodal personal agents use historical evidence when conversation state changes. The study keeps evidence, questions, and scoring constant while varying only the conversation state, then checks if agents can determine answerability, recover the correct source, and answer from it. Results across seven multimodal backbones show distinct failure regimes before and after compaction, heavy reliance on context continuity by open-weight models, and mixed effects of retrieval pressure on source attribution.

By Yu Liu, Wenxiao Zhang, Cheng Hu, Cong Cao, Fangfang Yuan, Xinyu Wang, Jin B. Hong, Yanbing Liu
arXiv AI
Sep 17

LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition

The paper introduces LM Fight Arena, a new benchmark that pits large multimodal models against each other in the fighting game Mortal Kombat II to evaluate real‑time visual understanding and sequential decision‑making. Six leading open‑ and closed‑source models were tested in a controlled tournament where each controlled the same character, ensuring a fair comparison. The framework offers a fully automated, reproducible, and objective assessment of an LMM’s strategic reasoning in a dynamic setting.

By Yushuo Zheng, Tongrui Ye, Zicheng Zhang, Xiongkuo Min, Huiyu Duan, Guangtao Zhai
arXiv AI
Sep 17

An Agentic Framework for Neuro-Symbolic Programming

The paper introduces AgenticDomiKnowS (ADS), a framework that converts free‑form task descriptions into fully functional DomiKnowS neuro‑symbolic programs. ADS employs an agentic workflow that builds and tests each DomiKnowS component independently, optionally allowing human‑in‑the‑loop refinement. The authors demonstrate that ADS enables both experienced and novice users to create complete neuro‑symbolic programs in 10–15 minutes, compared to the hour required for manual coding.

By Aliakbar Nafar, Chetan Chigurupati, Danial Kamali, Hamid Karimian, Parisa Kordjamshidi
arXiv AI
Sep 17

EvoUndo: Recoverability-Constrained Self-Evolution for LLM Agent Harnesses

EvoUndo is a framework that enables large language model agents to self‑evolve—modifying prompts, tools, and execution harnesses—while ensuring that these changes can be reliably reversed across different states. The study evaluates EvoUndo on 600 unseen one‑shot self‑evolution tasks, finding that 197 capability‑improving mutations fail recoverability checks. By extending the recovery language and adding exact state‑address diagnostics, the framework recovers up to 191 out of 197 failures, demonstrating that robust self‑evolution requires co‑designing verification, grounding, witness semantics, and recovery expressivity.

By Tanmay Sah, Dolly Sah, Harshul Jain, Tanya Sah
arXiv AI
Sep 17

After the Party: Growth, Governance, and Security Scanning in the OpenClaw Agent Skill Ecosystem

The paper examines the rapid growth of the OpenClaw AI agent’s public skill registry, noting a near doubling of the observable stock in 91 days and a concentration of activity in a short period. It finds that only a small fraction of skills receive significant attention—most have no stars or comments—while a large portion contains privilege‑evident code. Automated security scanners show low agreement and sensitivity, indicating that current tools are insufficient for reliable governance of fast‑expanding agent‑skill ecosystems.

By Yunpeng Xiong, Ting Zhang
arXiv AI
Sep 17

RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents

RideWay is a new benchmark that evaluates ride‑hailing language agents not just on task completion but on interaction efficiency. It introduces the Efficiency Utility metric, which penalizes agents for excessive tool calls and user‑facing turns relative to a task‑specific reference effort, with human preferences used to calibrate the penalties. Across 58 tasks and 24 models, the metric shows that extra dialogue is penalized more heavily than extra tool use, and it achieves high accuracy in distinguishing trajectories that differ in turns but struggles when differences are only in tool calls.

By Qingnuan Han, Boli Fang, Mingzhi Hou, Claire Liu