AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,733 stories · RSS feed

arXiv Computer Vision
Oct 23

RecallGS: Geometry-Grounded Recall for Online Semantic Gaussian Splatting

RecallGS is a lightweight, training‑free method for online semantic 3D Gaussian Splatting that recovers geometric‑semantic context from an agent’s observation history instead of learning an extra 3D network. It uses a recall‑ground‑fuse pipeline: CLIP retrieves relevant past observations, alpha‑compositing identifies geometrically supported views, and the semantic features of these views are aggregated over each Gaussian’s rendered footprint. Experiments on ScanNet and ScanNet++ show that RecallGS achieves strong semantic segmentation performance at 2.48 FPS.

By Thanh-Khoi Nguyen, Thien-Phuc Tran, Minh-Triet Tran
arXiv Machine Learning
Oct 23

Post-Audit Mirage: Identifiability Limits of Offline Agent Verification

The paper introduces the concept of a "Post‑Audit Mirage," showing that offline tests can fail to detect whether an agent update will remain safe once deployed. It formalizes this limitation using matched safe and harmful lifecycles that produce identical evidence to a verifier while an isolated scorer observes opposite outcomes, and presents a theorem describing the trade‑off between error and abstention. The authors also propose a benchmark and a tool called IdentifiedRangeMonitor that attempts to recover the missing distinction in certain settings, but notes that monitoring may still be undecided for rare groups, small effects, or missing observations.

By Aman Behera, Payal Payal
arXiv AI
Oct 23

Agent AI Zarz\k{a}dzaj\k{a}cy Sieci\k{a} Open RAN Zorientowana na U\.zytkownika

The article introduces an AI Agent designed to form user‑centric clusters in Open RAN networks. It proposes a language‑model‑based approach to address a key challenge in implementing user‑oriented Open RAN architecture. Simulation results show that the solution can more than double the median user throughput compared to a standard network‑centric architecture.

By Marcin Hoffmann
arXiv AI
Oct 23

Toward Recursive Self-Improvement in Physical AI

The paper discusses recursive self‑improvement (RSI) in physical AI, where a system not only enhances its behavior but also the methods it uses to improve. It reviews how robot learning can evolve both capabilities and improvement processes, highlighting that gains persist only when feedback addresses remaining learning needs, the component can apply the change, the value survives task or hardware shifts, and practice remains safe and affordable. Current physical studies mainly show behavioral and procedural gains, but have not yet isolated recursive benefits, leading the authors to propose evaluation principles and future research directions for safely trial‑by‑trial self‑improvement.

By Meng Luo, Zechen Bai, Shanqing Xu, Yanlin Li, Kaixin Li, Mengchen Qian, Weifeng Lu, Ran Zhang, Yingtian Zou, Ziyang Luo, Wei Chen, Hongzhan Lin
arXiv Machine Learning
Oct 23

AfterDeleteBench: Auditing Post-Deletion Information Persistence in LLM Agents

AfterDeleteBench is a controlled behavioral audit designed to test whether large language model agents retain information after a memory record is deleted. The benchmark compares agents that have been exposed to a synthetic secret and then had the original memory removed against matched agents that never saw the secret, attributing any post‑deletion recovery to the initial exposure. Results show that a single query can recover deleted secrets from models such as Qwen2.5‑7B‑Instruct in about two thirds of cases, and that reconstruction can succeed even when the full secret string is absent, with success rates of 62‑99% across Qwen, Mistral, and Phi‑3.

By Ehsan Saleh, Roozbeh Nahavandi, Sama Salarian, Raef Bassily, Andrew Perrault
arXiv Machine Learning
Oct 23

Lifetime-valid decisions by continually evolving AI analysts

The paper introduces a method called continual validation for AI analysts that evolve over time. By evaluating each new version on simulated null data before deployment, it guarantees that the lifetime probability of exceeding a target Type I error rate stays bounded, regardless of how many updates are made. Experiments on bioequivalence studies and A/B tests show that continual validation reduces Type I error rates from 0.098 and 0.183 down to at most 0.045 and 0.044 while preserving most of the statistical power.

By Xiaojing Sun, Yuhan Zeng, Xiao Wang
arXiv Machine Learning
Oct 23

SWE-VERIFIER: Selective Verification for Cost-Efficient Coding Agents

SWE-VERIFIER is a lightweight generative model that selectively triggers candidate sampling and verification for coding agents, scoring proposed actions to decide when additional sampling is needed and ranking the resulting actions before execution. Trained on Qwen3.5-4B with preference data, it improves resolution on MiniMax-M2.7 from 75.47% to 77.20% at 1.09× cost and on DeepSeek-V4-Flash from 72.60% to 74.27% at 1.13× cost, while verifying fewer than 4% of steps. The approach demonstrates that selective sampling and lightweight verification can enhance coding‑agent performance and inference efficiency with minimal overhead.

By Kaijian Zou, Vaibhav Chennareddy, Yunxiang Zhang, Naihao Deng, Longju Bai, Jiaxin Pei, Rada Mihalcea, Lu Wang
arXiv Machine Learning
Oct 23

Solvable models of learning to pursue a moving target

The paper introduces a continuous‑time linear‑quadratic model for agents learning to intercept a moving target, offering optimal control strategies that span reactive and predictive extremes. The model’s few interpretable parameters allow closed‑form solutions for optimal strategies, policy learning dynamics, and value learning dynamics. Analysis of the model’s eigenvalue spectra reveals a gap that explains why agents typically learn to reach the target before fully optimizing their movement.

By John J. Vastola, Kanaka Rajan
arXiv Machine Learning
Oct 23

What Lies Beyond Perception: A Partition Theory of the Limits of AI Evaluation and Oversight

The paper introduces a partition theory of AI evaluation, arguing that an evaluator’s ability to certify a model’s capabilities is limited by its perception. It shows that if a model’s input partition refines the evaluator’s, it can pass all tests the evaluator can grade, but only a stricter evaluator can raise the discrimination floor. The authors define an echo test that is complete for the evaluator’s tests, proving that passing it only guarantees a minimum discrimination level and cannot detect exact imitation of weaker models.

By Ahmed M. Mahran
arXiv Computer Vision
Oct 23

AgentVQA: A Unified Benchmark for Agentic Visual Understanding

AgentVQA is an offline benchmark designed to evaluate vision‑language models on realistic agentic scenarios across five domains: Web Agents, Robotics, Egocentric Videos, Games, and Spatial Planning. It comprises 1,000 carefully selected real‑world examples drawn from 23 source datasets, enabling reliable comparisons without costly online simulations. The benchmark reveals that current models achieve only 58.5% accuracy, with most errors stemming from difficulties in grounding and tracking evolving visual states rather than from task misunderstanding or reasoning.

By Advait Gupta, Sachin Konan, Tom Yin, Gabriel Sarch, Zhuang Liu
arXiv Computer Vision
Oct 23

BrickWright: Agentic Creation of Physically Feasible Brick Builds

BrickWright is a framework that enables coding agents to create physically feasible LEGO brick builds from text or image prompts. It provides a Python toolkit, BrickLayer, which includes an ergonomic assembly API and a validator that checks for real parts, collisions, floating parts, and connectivity. Using BrickWright, agents achieve 100% physical validity on 512 prompts and outperform baseline agents and specialized models on the text-to-brick task.

By Ava Pun, Kevin Dai, Sally Chen, Maxwell Jones, Ruixuan Liu, Kangle Deng, Changliu Liu, Maneesh Agrawala, Deva Ramanan, Jun-Yan Zhu
arXiv Computation and Language
Oct 23

The Value of Memory in Language Agents

The paper presents a decision‑theoretic framework for evaluating the value of memory in language agents. It tracks how memory transforms raw history into persistent state, operational use, and costly recovery or revision, expressing value in units of avoided task loss. The study explores risk‑cost frontiers, unknown demand profiles, interactive feedback effects, and provides simulations and counterexamples to illustrate the impact of information timing, retrieval access, component interaction, feedback, and demand uncertainty on memory’s effectiveness.

By Zehao Lin, Erenkai Zhu, Helan Hu, Xixuan Hao, Chunyu Li, Kai Chen
arXiv Computation and Language
Oct 23

Learning What to Ask for Tomorrow: Active Preference Acquisition for Long-Term Personalized Web Agents

The paper introduces Active Preference Acquisition (APA), a method for web agents to decide which unknown user preferences to ask about by evaluating the Prospective Personalization Value (PPV)—the expected cumulative benefit across future tasks minus the cost of asking. APA calculates counterfactual gains for each candidate preference using task prototypes and aggregates them by expected reuse, enabling agents to prioritize questions that may not help immediately but improve long‑term performance. Experiments on the ActiveWebPref benchmark show APA can acquire a large share of valuable preferences and achieve higher net utility than baseline strategies, demonstrating that considering future reuse is crucial for effective personalization.

By Yiqi Wang, Zizhi Zeng, Jiaqi Zhang, Yiqun Duan, Mingkai Zheng, Taotao Cai
arXiv Computer Vision
Oct 23

AnyGameRenderer: Entity-Consistent Generative Rendering Across Games and an Entity-Level Benchmark

AnyGameRenderer is a video‑to‑video generative renderer that transforms low‑quality, AI‑generated game footage—such as gray blockouts and colored primitives—into high‑quality visuals with detailed characters, motion, and effects while preserving layout, camera, and entity positions. It works across diverse game genres (tower defense, auto‑battlers, top‑down survivors, turn‑based battles, kart racing, crowd runners) and accepts plain video inputs without requiring complex engine data. Trained on ~10k paired clips from agent‑made games, Unreal Engine simulations, and real gameplay, it improves entity correspondence, raising temporal match from 50.9% to 62.9% and reducing missed count changes from 34.7% to 21.3%.

By Ruicong Liu, Liangyang Ouyang, Fengbo Lan, Zhixiang Wang, Kaipeng Zhang
arXiv Machine Learning
Oct 23

PriMa-Causa: Causal Prescriptive Maintenance

PriMa-Causa is a new framework for causal prescriptive maintenance that ranks maintenance actions by their estimated interventional effect. It combines a causal reasoning layer, a parametric simulator that generates interventional ground truth, and a graph-conditioned tabular causal foundation model trained once on synthetic production‑line data. The approach achieves lower prescription error than associational baselines, especially under weak overlap and sparse context, and shows further improvement when a partially disclosed causal graph is used.

By Felix Saretzky
arXiv AI
Oct 23

Fig: Automated Multiagent Behavior Elicitor

Fig (Finding Interactions in Games) is an agentic system that performs targeted elicitation interventions on game environments to study multiagent behaviors. It takes a natural‑language behavior description, decomposes it into sub‑behaviors, generates or selects appropriate social games, and applies interventions to the game engine, opponents, instructions, and context. The system evaluates model behavior using mechanical checks and synthetic rubric judgments, and human review validates the interventions, generated games, and evaluation mechanisms. In experiments, models attempted the target behavior in 45.2% of episodes on average for six test behaviors, and Fig’s interventions increased attempt rates by 10.9% compared to matched control games.

By Anrui Gu, Zhichao Wang, Nicolai Ouporov
arXiv AI
Oct 23

Forging More Tasks into Better Harnesses: Agentic Harness Evolution via Successive Scale Racing

The paper introduces Successive Scale Racing (SSR), a framework that improves agentic harness evolution by adaptively selecting optimal evidence combinations during each evolution round. SSR constructs diverse evidence sets based on task quality and coverage, then efficiently races candidate harnesses under a limited compute budget. Across three benchmarks, SSR outperforms baselines, steadily raising pass rates as the task pool expands from 40 to 200 while cutting validation executions by 60%.

By Jingzhe Liu, Jiliang Tang, Tong Zhao, Neil Shah, Clark Mingxuan Ju
arXiv AI
Oct 23

Back Up Before You Regret: Measuring Backup Awareness and Automating Backup in Coding Agents

The paper introduces RegretBench, a benchmark of 205 tasks that capture moments where users regret not backing up state, and evaluates six advanced coding agents on their backup awareness. While agents complete 86.1% of tasks, they only back up before regret moments 15.2% of the time, revealing a gap between task success and state preservation. To address this, the authors propose BackUpper, a plug‑in that automatically preserves affected state before tool calls, achieving 92.2% similarity to the original state on RegretBench at a cost of $0.014 per task and aligning with user preferences with an 84.5% F1 score.

By Xu Li, Simon Yu, Minzhou Pan, Yiyou Sun, Zhaorun Chen, Bo Li, Weiyan Shi
arXiv AI
Oct 23

D-Skill: Distilling Teacher Knowledge through Agent Skills

D‑Skill is a framework that distills the behavioral advantage of a large language‑model teacher into an interpretable natural‑language skill document, using only paired rollouts and no weight updates. The skill is constructed offline from fixed trajectories, then provided to a smaller student model at inference time. Across benchmarks such as SpreadsheetBench‑Verified, ALFWorld, and chess puzzles, D‑Skill closes 75‑95% of the teacher‑student performance gap and reduces inference cost by 5.7×–8.3× compared to the teacher.

By Shuo Yuan, Dacheng Li, Joseph E. Gonzalez
arXiv AI
Oct 23

PreCoMem: Predictive Cognitive Memory for Self-Evolving Long-Term Dialogue Agents

PreCoMem introduces a memory consolidation framework for long‑horizon dialogue agents, inspired by predictive coding. It represents user memory as confidence‑weighted beliefs and uses an Effective Surprisal signal to decide whether to maintain, profile, or correct each belief. Experiments on LoCoMo, LongMemEval‑S, PersonaMem‑v2, and the new DiMoBench demonstrate state‑of‑the‑art accuracy and robust adaptation to non‑stationary belief drift without over‑reacting to noise.

By Jian Zhong, Zeyu Liu, Pingchuan Cao, Rongduo Han, Shunye Tang, Guohuan Xie, Mingyuan Qin, Ran Ji, Xiao Liang, Wei Wang, Haining Zhang