AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,743 stories · RSS feed

arXiv Computer Vision
Sep 25

CoaG: Cylinders on a Grid for Coarse 3D Layout Control in Video Generation

The paper introduces CoaG, a method that allows users to control the placement and movement of people and the camera in generated videos using minimal geometric input—a ground plane grid and one cylinder per person. By drawing a grid, positioning cylinders, and animating them over 81 frames, the model produces photorealistic videos that respect the specified layout and motion, while appearance is guided by a text prompt and a background image. To train the system, the authors automatically generate 2,000 caption–video pairs, extract geometry via tracking and reconstruction, and fine‑tune a LoRA on a text‑to‑video model to follow user‑drawn layouts and camera paths.

By Zhangsihao Yang, Mengyi Shan
arXiv AI
Sep 25

Who Holds the Pen? Let Specifications, Not Agents, Sign Off

The paper argues that large language model agents should not be the sole authority on whether they have satisfied a task. It identifies two gaps—understanding–execution and state–authority—where agents may claim completion without actually meeting the specification. The authors propose SpecHarness, a framework that separates agent proposals from authoritative state by requiring evidence from qualified providers to confirm compliance, and demonstrate its effectiveness on guideline‑following and artifact‑generation tasks.

By Haiqing Li, Xin Ma, Yinhao Wu, Wenliang Zhong, Feng Jiang, Thao M. Dang, Xiao Hu, Hehuan Ma, Yuzhi Guo, Junzhou Huang
arXiv Machine Learning
Sep 25

Temporal Gradient Inversion for Private Trajectory Reconstruction in Embodied Reinforcement Learning

The paper introduces TRACE, an amortized temporal gradient‑inversion attack that reconstructs private observation‑action trajectories from per‑step policy gradients in embodied reinforcement‑learning agents. TRACE exploits cross‑time correlation between gradients and exact action recovery from policy‑head gradients, achieving high reconstruction quality (18.8 dB PSNR) and near‑perfect action recovery with minimal computation. The study demonstrates TRACE’s effectiveness across various neural architectures and input modalities, and suggests that protecting temporal gradient streams may require sequence‑aware privacy mechanisms.

By Sudip Bhujel, Shanghao Shi, Ruiquan Huang, Ning Zhang, Yang Xiao
arXiv AI
Sep 25

Pistis Technical Report

The Pistis Technical Report introduces the Pistis model family, comprising 27B- and 9B-parameter multimodal large language models built on Qwen3.6 and Qwen3.5. The models are developed through a scalable post‑training framework that begins with large‑scale multimodal supervised fine‑tuning and then applies Interleaved Distillation and Reinforcement Learning (IDRL) to integrate on‑policy distillation and reinforcement learning within a single training loop. Two specialized variants—Pistis‑Thinking for deep multimodal reasoning and Pistis‑Agentic for long‑horizon planning, iterative reasoning, and tool use—are produced at both scales, and a system‑level method called Pistis‑Auto‑Harnessing (PAH) further improves inference harness performance without updating model parameters.

By Heyun Chen, Xiaohan Lan, Jiaxi Li, Zhilin Lu, Qi She, Weiwen Xu, Fei Yu, Yujie Zhong, Jinghuan Chen, Zijian Feng, Siyu Jiao, Yiheng Lin, Xinhao Wang, Sihan Yang, Jieyu You, Changbin Zhang, Hengyu Zhang, Xudong Zhang, Yunqing Zhao, Shuai Zheng
arXiv AI
Sep 25

RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers?

RECLAIM is a benchmark that tests whether AI agents can reproduce results from 100 NeurIPS 2025 papers, with each paper pre‑defined with a target result, success criteria, and GPU‑hour budget. Papers are categorized into three difficulty tiers—Run, Retrain, and Reimplement—based on the availability of code, data, and weights. In experiments, the best agent succeeded on only 41% of Run‑tier papers, 27% of Retrain‑tier, and 15% of Reimplement‑tier, often stopping early with unused budget and frequently making errors such as writing code without verifying against the paper’s numbers.

By Mithil Salunkhe, Haochen Ding, Samridhi Verma, Volodymyr Kindratenko
arXiv AI
Sep 25

Scope Before You Persist: Preventing Cross-Family Interference in Agent Memory

The paper introduces a method called Scoped-ORC that matches the retrieval scope of persistent memory edits to the certification scope of language‑model agents. By retrieving each accepted skill only for its originating task family, the approach raises mean hidden trajectory utility from 0.713 to 0.816 and eliminates harmful deployments in a 12‑round code‑repair stream. Across 27 randomized streams, Scoped-ORC improves trajectory utility by 0.063, accepts more updates, and records no harmful acceptances, demonstrating that scope matching complements certification to prevent cross‑family interference in agent memory.

By Yezhou Cheng, Runjia Du, Zeming Liu, Qibai Chen, Hang Lyu, Yankai Zeng, Yilan Wei, Bojun Lin
arXiv AI
Sep 25

IndicBankBench: Evaluating Safety and Reliability of Language Model Assistants in Indian Retail Banking

IndicBankBench is a 799‑case benchmark designed to evaluate the safety and reliability of language model assistants in Indian retail banking. It covers five operational domains, a capability/refusal domain, and twenty primary axes, assessing each case at four stages: safety, action and tool use, response adequacy, and advisory quality. The benchmark uses deterministic safety checks, a narrow resolver for ambiguous confirmation‑before‑write scenarios, and an LLM judge for semantic response adequacy, reporting strict pass rates that reveal a gap between strict reliability (43.7%–58.2%) and at‑least‑once success (60%–74%).

By Suvradip Paul, Chandra Bhushan, Harsh Sharma, Nitin Kukreja, Yatharth Dedhia, Keyur Doshi, Prashant Devadiga
arXiv AI
Sep 25

Towards An LLM-Driven Unified Conversion Framework for BT and FSM in Autonomous Intelligent Systems

The paper introduces an LLM-driven framework that automatically converts between finite state machines (FSM) and behavior trees (BT) for autonomous intelligent systems. It addresses key challenges such as preserving behavioral completeness and preventing model complexity explosion by designing a loop execution BT structure and employing depth compression strategies with LLM prompts. Experiments in various autonomous decision-making scenarios show that the framework achieves accurate, scalable, and maintainable bidirectional conversion, benefiting consumer-grade applications like service robots, game agents, and smart home devices.

By Zhang Qi, Yang Shuo, Zhu Zhengqiu, Zhou Peng, Jiao Peng
arXiv AI
Sep 25

SkinAgent AI: A Safety-Grounded Multimodal Agentic Framework for Non-Diagnostic Skincare Support

SkinAgent AI is a multimodal, safety‑grounded framework designed for non‑diagnostic skincare support. It routes visual concerns (Acne, Pores, Wrinkles), estimates skin type from photographs, and provides count‑informed acne severity, all while grounding recommendations in a database and enforcing deterministic safety, privacy, and evidence checks. The system achieved high accuracy in routing and skin‑type estimation, and demonstrated no safety or privacy violations in controlled tests, though tool‑selection errors and incomplete product grounding remain.

By Muhammad Muhtasim Shahriar, Abdullah Mohammad Sayem, Tze Hui Liew, M. F. Mridha, Md. Mahiuddin
arXiv AI
Sep 25

Delay-of-Gratification as a Multi-Agent Survival Micro-benchmark for Long-Horizon LLMs: Social Exposure, Personas, and Tool Use Budgets

The paper introduces a new multi‑agent micro‑benchmark called Delay‑of‑Gratification, modeled after the Stanford marshmallow experiment, to evaluate large language models (LLMs) in long‑horizon, multi‑turn interactions. In the benchmark, ReAct agents use a per‑step “raise a question” tool under various constraints—social context (broadcast vs. isolated), persona traits (age, hedonic drive), and tool‑use policy (mandatory vs. optional). Across 19,200 trajectories, the study finds that most agents exhibit an early impulse to “eat,” only 75.9% persist to the end, and factors such as isolation and hedonic drive significantly influence survival and questioning behavior, with ablations showing that removing hedonic drive and age can improve completion rates.

By Olga Manakina, Igor Bogdanov, Chung-Horng Lung
arXiv AI
Sep 25

ERRAND: Budgeted Maintenance of Agent Memory

ERRAND is a new method for budgeted maintenance of agent memory that treats revalidation of stored knowledge as a priced errand competing for scarce actions. It uses an errand index that is single‑peaked, allowing certainty in either direction to cost nothing, and repairs by writing new versions rather than deleting old ones. In experiments across two drifting tool‑use worlds, ERRAND outperforms non‑oracle policies, achieving up to 10.0 percentage points improvement over eager revalidation while using only 11.0% of steps, and it self‑terminates when no budget is imposed.

By Beining Wu, Zihao Ding, Jun Huang
arXiv AI
Sep 25

Ingest-Time Fact Compilation for Cost-Efficient and Reliable Question Answering over Revised Corpora

The paper introduces ingest‑time fact compilation, an architecture that preprocesses and compiles corpus data into self‑contained facts with resolved revisions, deletions, and source trust. By storing this compiled state, query‑time models can retrieve answers directly, avoiding costly reconstruction from raw passages. Experiments show that this approach reduces read cost per question by 12.89× and token usage by 21.6× while maintaining accuracy.

By Kyle Wild, Yusuke Takahashi, Asako Uraki
arXiv AI
Sep 25

AI-based detection of worsening heart failure from low-resolution telemonitoring data

The study introduces TRACER, a Transformer-based model that uses contrastive event representation to predict timelines leading to heart failure hospitalizations from low-resolution, irregularly sampled telemonitoring data. TRACER incorporates time-aware embeddings, contrastive pre‑training for anomaly detection, and independent binary classifiers, and was evaluated on biomarker sequences from 276 heart failure patients. The model achieved 66.7% accuracy in predicting hospitalization timelines with a 7.9% overestimation, outperforming other tested models by reformulating training as an event detection problem.

By Erik Aerts, Yinan Yu, Annika Rosengren, Michael Fu, Martin Lindgren, Falk Dippel, Martin Adiels, Helen Sj\"oland
arXiv AI
Sep 25

PUBG Ally: A Conversational Embodied Agent as an AI Teammate

PUBG Ally is an embodied, voice‑enabled AI teammate for PUBG: BATTLEGROUNDS that can perceive the game world, interpret player speech, and autonomously decide actions while keeping speech synchronized with gameplay. It combines a language‑model agent that uses a controlled interface to gather game information and a faster control layer for movement, combat, and recovery. The system was trained on nearly 39,000 real‑player sessions and evaluated through player feedback and preference comparisons, with live deployment requiring low‑latency on‑device execution and safety safeguards.

By Beomsoo Kim, Byeongju Kim, Dohyun Kim, Dongwon Kim, Eunchong Kim, Hongmin Kim, Hyeojung Im, Hyeonbin Hwang, Hyeonghwan Kim, Hyoseok Seol, Insub Im, Irene Chen, Jaeseung Jeon, Jimin Hong, Kiyoon Yoo, Minkyoung Park, Seohyeon Jung, Seungjun Chung, Sue Hyun Park, Sungwoo Kim, Youngin Cho, Yujeong Son, Kangwook Lee, Hyunseung Kim
arXiv AI
Sep 25

When Can Agents Forget Their Reasoning? ICLR for Long-Horizon Agent Context Compression

Long‑horizon language model agents accumulate reasoning history, which inflates context length and inference cost. The paper introduces Interaction Aware Compression for Long Horizon Reasoning (ICLR), a training‑free online method that ranks and removes reasoning blocks based on frozen proxy entropy while preserving actions, tool calls, and observations. On 260 WorkBuddyBench tasks, ICLR raises average reward from 0.699 to 0.718 and cuts input, output, and cache read tokens by 25.5%, 14.4%, and 33.3% respectively, while analyses show that historical reasoning becomes replaceable once task‑relevant state is externalized.

By Mingxuan Wang, Fei Luo, Bo Wang, Guorun Yao, Yinglong Guo, Chao Ning, Hongyue Chen, Yanbiao Ma, Jungong Han
arXiv AI
Sep 25

How does Adversarial Influence Scale in Multi-Agent Systems?

The paper investigates how deception affects multi‑agent deliberation, finding that the key factor is the proportion of deceivers rather than the total number of agents. Defection rates—instances where initially correct agents adopt incorrect conclusions—grow linearly with the deceiver proportion, and large language model agents are vulnerable even when deceivers are a minority. The study also shows that coordination among deceivers can reduce their effectiveness and that the specific models involved influence susceptibility.

By Addison J. Wu, Jasin Cekinmez, Michel Liao, Karthik Narasimhan, Thomas L. Griffiths
arXiv AI
Sep 25

Jev-Mobile: Jev as an Executor for Mobile GUI Agents

Jev-Mobile introduces a new approach for mobile GUI agents by separating high‑frequency lightweight execution from low‑frequency vision‑language model (VLM) planning. The VLM sets local goals, the accessibility tree provides a structured action space, and Jev, a fast typed decision model, repeatedly selects actions within this space, allowing multiple GUI actions per VLM decision. On the AndroidWorld task suite, Jev-Mobile achieves 79% task success, reduces mean end‑to‑end execution time by 32.7%, and cuts model API cost by 73.4% compared to a step‑wise VLM baseline.

By Linghua Zhang
arXiv AI
Sep 25

Certified Task-Conditioned Active Observability

The paper introduces the concept of task‑conditioned active observability, defining the minimal interaction cost needed for an autonomous agent to identify task‑relevant states while guaranteeing safe abstention. It formalizes this complexity, proving that task‑predictive equivalence yields a unique minimal sufficient quotient that preserves complexity and eliminates unnecessary distinctions. The authors present theoretical characterizations for deterministic and noisy regimes, and demonstrate a certified observer that reduces sensor usage and model steps while maintaining zero false acceptances in extensive high‑dimensional trials.

By Linzhe Zhang, Changming Xu
arXiv AI
Sep 25

Who Is Behind the Harness? Fingerprinting LLMs through Agentic Behavior

The paper introduces LIDAR, an active black‑box fingerprinting technique that identifies large language models (LLMs) by observing their decisions and actions during coding‑agent execution. By probing agents on tasks such as post‑edit verification, transient‑failure recovery, and specification–test conflict resolution, LIDAR captures both instance‑level and distribution‑level features of execution trajectories and compares them to clean references using a lightweight probabilistic identifier. Evaluated on 36 models across seven families and two agent harnesses, LIDAR achieves high Top‑1 accuracy and MRR, outperforming four existing fingerprinting and API‑auditing baselines, and demonstrates that agent execution behavior provides model‑identity evidence beyond final outputs.

By Chuyi Wang, Xiaohui Xie, Tongze Wang, Fangchen Luo, Yong Cui
arXiv AI
Sep 25

Where Cyber Agents Struggle: Bottleneck Analysis of Multi-Stage LLM Agents

The paper presents a diagnostic study of a multi‑stage LLM‑based cyber agent system, examining its orchestrator, executor, and validator components in enterprise‑style lateral‑movement scenarios. Six advanced LLMs were tested across expert‑defined, self‑scaffolded, and fully autonomous modes, with metrics that include validator consistency, evidence grounding, token usage, retries, and runtime. Findings show that while validators are generally relevant, they are often nonspecific and overly optimistic, and the main bottlenecks lie in credential acquisition and lateral‑movement tasks, especially under full autonomy.

By Saeedeh Lohrasbi, Mohammad Mamun, Ahmed Yehia, Scott Buffett, Sherif Saad