AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,750 stories · RSS feed

arXiv Machine Learning
Sep 7

KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU

KVMem is a KV-context virtualization system that allows large language model agents to maintain workspaces exceeding both GPU key‑value capacity and the model’s native context window. It stores overflowed history as paged KV state across GPU memory, host memory, and NVMe, using lightweight, model‑native attention‑space indexes to retrieve relevant historical blocks. Evaluations on long‑context agent benchmarks show that KVMem improves task utility and inference efficiency, enabling up to one million‑token workspaces on consumer GPUs and achieving interactive responsiveness in local deployments.

By Di Chai, Leye Wang, Zeshen Su, Zhiguo Xia, Zhihang Yu
arXiv Machine Learning
Sep 7

How to Speculate about Uncertainty in Agentic Coding? A Draft-Model Gate Method

The paper introduces Speculative Uncertainty (SU), a technique that infers a failure likelihood for black‑box LLM agents by evaluating their generated token sequences with a lightweight draft model, without needing internal model details. SU extracts phase‑aware features from reasoning and action spans, calibrates them against verifiable outcomes, and produces a failure‑likelihood score usable by downstream policies. Applying a pre‑execution veto gate based on SU to software‑engineering agents such as Qwen3‑Coder‑480B and Claude 3.5 Sonnet reduced execution error rates by 6‑8 percentage points and token costs by 14‑19 %, while maintaining performance on out‑of‑distribution benchmarks and across different agent models.

By Konstantin Grotov, Valentin Malykh
arXiv Computation and Language
Sep 7

EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?

EVOHARNESSBENCH is a new benchmark that tests how LLM-based agents handle changes in their tool, skill, and agent harnesses over time. It includes 17 deterministic harness streams with 802 tasks, 520 tools, 42 skills, and 62 agents, and evaluates agents in two settings: deployment evaluation and self‑evolving adaptation evaluation. The study finds that harness expansion can cause forgetting, adaptation gains are inconsistent, and preserving old competence does not always aid new capability adaptation, highlighting harness evolution as a distinct challenge for agent development.

By Zixuan Ke, Vaidehi Patil, Haizhou Shi, Yang Li, Ye Liu, Sarath Shekkizhar, Anurag Koul, Jiayu Wang, Xuan Phi Nguyen, Semih Yavuz, Mohit Bansal, Shafiq Joty
arXiv Computer Vision
Sep 7

Efficient Multi-Timescale Event Representations for Feed-Forward Object Detection

The paper introduces a confidence‑normalized continuous multi‑timescale representation for event cameras, using logarithmic B‑spline temporal encoding and a geometry‑aware local confidence mechanism. When paired with a fixed feed‑forward EventCenterNet detector, this representation outperforms the compact CSTR representation on the PEDRo and Gen1 datasets. Additionally, a recursive exponential‑polynomial approximation is proposed to allow efficient event‑by‑event updates while maintaining detection performance.

By Fredrik Lundell, Per-Erik Forssen, M{\aa}rten Wadenb\"ack, Astrid Lundmark
arXiv Computer Vision
Sep 7

Cross-Domain Tracker Adaptation Without Target-Domain Labels via Vision-Language Agents

The paper introduces a Vision‑Language Model (VLM) that acts as a diagnostic agent to adapt a detect‑to‑track system to new domains without target‑domain labels. By inspecting rendered tracking outputs, the VLM identifies failure modes and iteratively recommends parameter updates, recovering a significant portion of performance lost when transferring hyperparameters from a source domain. Experiments on MOT17→MOT20 show the VLM tuner restores 67.8% of the lost headroom, while Bayesian optimization with proxy objectives performs poorly under large domain shifts.

By Daniel Davila, Ravikumar Balakrishnan, Mike Cochran
arXiv Computer Vision
Sep 7

AquaBEV: Monocular Underwater BEV Occupancy with 3D Sonar Supervision

AquaBEV is a monocular underwater occupancy model that predicts local bird’s‑eye‑view (BEV) occupancy from a single RGB image. It uses paired 3D imaging sonar data as geometric supervision during training, mapping visual features into a calibration‑free polar representation and decoding along the range dimension before reconstructing Cartesian BEV coordinates. In a controlled underwater occupancy benchmark, AquaBEV outperforms the strongest transferred baseline with 31.4 % Visible IoU and 38.6 % Observed IoU, achieving 4.0 % and 4.3 % relative improvements respectively.

By Trung Tien Dong, Shengji Jin, Chen Chen, Yi Sheng, Xiaomin Lin
arXiv Computer Vision
Sep 7

CoLMIN: LLM-based Multi-Decision Path Negotiation for Cooperative Autonomous Driving

CoLMIN is an LLM-based framework for cooperative autonomous driving that addresses premature convergence to suboptimal solutions in multi-solution traffic scenarios. It introduces a Multi-Intent Negotiation module that generates multiple candidate driving intentions, an Evaluation-based Shallow Reflection Module that provides feedback to accelerate consensus, and a Deep Reflection Module that mitigates cognitive fixation by reflecting on negotiation histories. Experiments in the CARLA simulation show that CoLMIN outperforms existing methods in challenging interactive driving scenarios.

By Zhe Huang, Zhaoxin Fan, Shuo Wang, Wenjun Wu, Xuan Zhao, Min Liu
arXiv Computer Vision
Sep 7

Out-of-Distribution Semantic Occupancy Prediction

The paper introduces Out-of-Distribution Semantic Occupancy Prediction, a task that focuses on detecting unknown objects in 3D voxel space for autonomous driving. It proposes Realistic Anomaly Augmentation to create two new datasets, VAA-KITTI and VAA-KITTI-360, and presents the OccOoD framework, which uses Cross‑Space Semantic Refinement to improve OoD detection while maintaining semantic occupancy accuracy. Experiments show OccOoD achieves an AuROC of 65.50% and an AuPRCr of 31.83% within a 1.2 m radius, demonstrating strong generalization to real‑world urban scenes.

By Yuheng Zhang, Mengfei Duan, Kunyu Peng, Yuhang Wang, Ruiping Liu, Fei Teng, Kai Luo, Zhiyong Li, Kailun Yang
arXiv Statistics ML
Sep 7

Towards AI-Driven Nanomedicine Discovery: A Benchmark and Multimodal Learning Framework for Nano Self-Assembly Prediction

The paper introduces NSA-Bench, a public benchmark for predicting nano self‑assembly (NSA) between molecular pairs, framing it as a binary classification problem. It presents NSA‑Net, a multimodal learning framework that fuses graph topology, sequence semantics, and physicochemical descriptors to predict self‑assembly, achieving high ROC‑AUC scores and outperforming existing baselines. The study also demonstrates how NSA‑Net’s predictions can guide experimental formulation refinement through an NSA‑Agent case study.

By Quan Hao, Mengyue Fan, Zifan Dong, Jianduo Zhao, Changhao Xiao, Shangqing Jiao, Hao Zhang, Yudong Wang, Fei Xia, Jigang Wang, Liguo Zhang, Chong Qiu
arXiv AI
Sep 7

Iris: Climbing to the Search Frontier

The paper introduces Iris-mini and Iris-pro, two search agents trained at 35B and 397B parameter scales. They use a novel data pipeline that constructs reverse‑engineered multi‑hop queries from web hyperlinks, filters trajectories, and alternates supervised fine‑tuning with reinforcement learning in a process called SFT‑RL climbing. Evaluations on several benchmarks show that, with inference‑time context management, the agents achieve the best open‑source results in their parameter ranges.

By Ziyuan Liu, Hengqi Liu, Zichuan Wang, Yang Qin, Jiachen Liang, Xu Chu, Shaowei Chen, Yuantao Gu, Mu Chuan
arXiv Machine Learning
Sep 7

Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving

The paper investigates how prefix caching, a default optimization in open‑source LLM serving stacks, affects reproducibility when combined with weight quantization. Experiments on an eighty‑episode multi‑turn agentic tool‑use workload show that enabling the cache causes the agent’s trajectory to change in 36.2 % of episodes at 16‑bit precision and 75.0 % at 4‑bit precision, while disabling the cache yields perfectly reproducible runs. The study identifies specific cache‑related settings that drive run‑to‑run divergence and demonstrates that cached serving is deterministic only when the cache state is preserved, which is not the case in typical deployments.

By Aditi Patodiya
arXiv AI
Sep 7

IPGeoAI: Transformer-Based Geolocation with LLM Semantic Fusion

IPGeoAI is a transformer-based deep learning model that transforms IP geolocation from a static lookup into a sequential modeling task. It captures hierarchical dependencies in IP subnet structures and resolves geographic ambiguity by fusing unstructured semantic context extracted from Autonomous System descriptions via a zero‑shot LLM feature extraction pipeline. Extensive offline evaluation on a proprietary dataset of 200,000 cities shows IPGeoAI outperforms a leading vendor, achieving a 6% improvement in city‑level accuracy and 100% traffic coverage, while online production tests demonstrate a statistically significant +0.35% improvement in first‑tier downstream use‑case metrics.

By Avinash Kadimisetty, Andy Jinqing Yu, Philip Favaloro, Wenlong Liu, Xiaolu Xiong
arXiv AI
Sep 7

From Language Models to World-Acting Systems: Progress and Limits of Agentic AI across Digital, Social, Virtual, and Physical Environments

The paper reviews how large language models have evolved into agents that can influence external environments through tool use, interface operation, delegation, state retention, virtual world inhabitation, and robotic control. It critiques the narrative of a single march toward autonomy, distinguishing model competence from system integration, persistence, and safe authority. The authors find that action-interface expansion is well documented, while robust completion, recovery, authorization, and independent verification remain less proven, and they propose a framework of justified delegation to guide future research.

By Linsen Zhu, Mengqing Cai
arXiv AI
Sep 7

Substrate-Aware AI Agents: Execution Context as a First-Class Input

The paper introduces the concept of substrate blindness, where AI agents lack execution context in their planning. By providing a 128 MB RAM and 10 s wall‑time contract to large language models, the authors show that agents generate code that uses less memory, runs faster, and incorporates structural changes such as bounded blocking and in‑place buffers. Across three leading models, contract disclosure improved resource usage and correctness, demonstrating that minimal execution contracts can guide agents to produce more efficient programs.

By Manu Agrawal
arXiv AI
Sep 7

Improving Weak World Models Behind Strong Agents in Atari Pong

The paper investigates the discrepancy between strong agents and their underlying weak world models in Atari Pong by reproducing five visual world-model agents and evaluating their frozen models. Closed‑loop rollouts reveal visual and dynamical failures such as ball disappearance and incorrect motion, while zero‑shot model‑based RL policies trained entirely within the frozen models perform poorly compared to the original agents. To address these issues, the authors introduce Concept‑Guided Spatial Regularization (CGSReg), an auxiliary loss focused on task‑critical ball regions, which improves both pixel‑space zero‑shot MBRL performance and closed‑loop rollouts for several agents.

By Yukuan Lu, Zaishuo Xia, Weyl Lu, Yubei Chen
arXiv AI
Sep 7

YOLO with Kolmogorov-Arnold networks and vision-language foundation models for interpretable object detection with trustworthy multimodal AI in computer vision perception

The paper introduces a Kolmogorov-Arnold network as an interpretable post‑hoc surrogate to assess the trustworthiness of YOLOv10 object detections, using seven geometric and semantic features. Its additive spline structure allows direct visualization of each feature’s influence, revealing when confidence scores are reliable or unreliable. A bootstrapped BLIP language‑image model generates scene captions, providing a lightweight multimodal interface that does not compromise interpretability. Experiments on COCO and University of Bath campus images show the framework accurately flags low‑trust predictions under blur, occlusion, or low texture, offering actionable insights for risk mitigation.

By Marios Impraimakis, Daniel Vazquez, Feiyu Zhou
arXiv Computer Vision
Sep 7

VoxelFix: Post-Hoc Semantic Correction of Completed 3D Voxel Maps

VoxelFix is a graph‑based post‑hoc semantic correction method that refines voxel labels in completed 3D voxel maps while preserving their geometry and occupancy. It learns to correct errors by exploiting local geometry and neighboring semantic information, using training pairs generated by corrupting annotated maps with class confusions from upstream perception pipelines. Experiments on OccuFly maps show consistent improvements of 4.23–5.00 percentage points in mIoU, especially for tree, roof, and wall classes, and the method generalizes to out‑of‑distribution aerial scenes.

By Sunesh Praveen Raja Sundarasami, Taehyoung Kim, Johannes Scherer, Toma\v{z} Coti\v{c}, Sivasubiramaniam Subbiah, Andreas Greiner, Paul Spannaus, Sebastian Houben
arXiv AI
Sep 7

Rethinking Indirect Prompt Injection as a Test-Time Search Problem

The paper redefines indirect prompt injection as a test-time search problem, framing it as exploration over a task-dependent attack surface shaped by the environment, user task, and injection task. It introduces an agentic attacker equipped with a search harness that conducts environment reconnaissance, structured reasoning on attack strategies, and adaptive evaluation using victim-agent feedback. Experiments across diverse tasks show that higher attacker compute leads to better vulnerability discovery, and that managing search strategies is crucial to avoid redundancy and maintain gains at larger budgets.

By Duong M. Nguyen, Joon Sik Kim, Blazej Manczak, Vaikkunth Mugunthan
arXiv AI
Sep 7

DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems

The paper introduces DCFA, a training‑free framework for attributing failures in large language model‑based multi‑agent systems. DCFA uses a global module to build causal‑inspired dependency graphs from system traces, pinpointing the earliest decisive error, and a local module that refines this attribution through counterfactual reasoning. Experiments on the Who&When benchmark across six LLMs demonstrate that DCFA improves step‑level accuracy by up to 8.27% over existing baselines.

By Zehao Wang, Lanjun Wang, Shilong Jin, Junjie Chen, Yanghua Xiao
arXiv AI
Sep 7

CoSkill: Joint Reinforcement Learning of Reasoning and Meta-Skill Agents for Hierarchical Skill Evolution

CoSkill introduces a unified multi‑agent reinforcement learning framework that jointly trains a Reasoning Agent and a learnable Meta‑Skill Agent over a hierarchical skill library. By treating the meta‑skill workflow as a trainable agent and sharing a single backbone, CoSkill enables end‑to‑end co‑adaptation, allowing the Reasoning Agent to condition actions on retrieved task and step skills while the Meta‑Skill Agent refines those skills based on task performance. Experiments on ALFWorld and WebShop demonstrate that CoSkill outperforms prior skill‑based and RL baselines, achieving higher success rates and improved sample, asymptotic, and wall‑clock efficiency.

By Jinyuan Feng, Dongmin Li, Yiqun Chen, Yang Gao, Xing Chen, Huimu Wang, Zhiqiang Pu