AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,743 stories · RSS feed

arXiv Computer Vision
Oct 9

Fluid-Gen-Zero: Grounding Pretrained Video Generators in Physics without Training

Fluid-Gen-Zero is a training-free framework that generates physics-aware fluid-object interaction videos by separating physical reasoning from appearance synthesis. It uses a physics simulator for motion dynamics and a pretrained video generator for visual quality, coordinated through a two-level agentic workflow involving a vision-language model for planning and latent-space guidance for simulation signals. The method improves simulation alignment and fluid flow accuracy across multiple backbones while maintaining perceptual quality, as shown by quantitative metrics and human preference studies.

By Hong Huang, Yuqiu Liu, Chenyu You, Daniel Martin, Chuhang Zou, Wuyang Chen
arXiv Computer Vision
Oct 9

No Distillation Needed: Single-Pass Real-Time Talking Heads via Acausal Noise Shaping

The paper introduces FaceGAN, a single-pass GAN that generates real-time talking head animations from audio without the need for distillation or diffusion models. By shaping the noise pathway acausally, FaceGAN maintains a causal audio-to-expression mapping while suppressing unwanted spectral artifacts, enabling continuous, drift-free generation at interactive rates. The approach achieves comparable or superior quality to state‑of‑the‑art methods while requiring only one forward pass per frame.

By Yu Han, Dejan Markovic, Alexander Richard, Wojciech Zielonka, Akshay Venkatesh, Cheng-hsin Wuu, Michael Zollhoefer
arXiv AI
Oct 9

Deconstructing Off-Policy Ratios: Entropy-Normalized Trust Regions for Asynchronous Reinforcement Learning

The paper introduces Entropy‑Normalized Trust Region (ENTR), a method for asynchronous reinforcement learning that adjusts off‑policy ratio thresholds based on token entropy rather than a single magnitude cut‑off. By recognizing that the natural scale of the ratio is set by entropy, ENTR preserves genuine exploration while filtering out noise from low‑entropy, stale data. Experiments on long‑horizon agentic tasks and mathematical reasoning benchmarks show ENTR outperforms existing asynchronous methods, improving BrowseComp‑Plus performance by 6.9 % and enabling stable training up to 30 policy versions of staleness while matching synchronous GRPO at a 2.6× speedup.

By Guanqun Zhao, Zijun Xie, Binbin Zheng, Yehan Yang, Jiafeng Lu, Aoqi Hu, Enlei Gong, Zeyu Chen
arXiv AI
Oct 9

Can LLMs Fix It Without Code? Toward Automated Verification of No-Code Bug Fixes

The paper investigates whether large language models (LLMs) can automatically verify no-code bug fixes—solutions that instruct users to change settings, upgrade versions, or alter workflows—by executing them in a real browser environment. Using 322 fixes from a benchmark, the study tests three executor agents (OpenCUA‑72B, Claude Sonnet 5, and Meta’s Muse Glimmer) and finds that only 17.6% of issues could be set up and passed sanity checks, with resolution rates ranging from 14.6% to 49.7% depending on the executor. Even the best configuration resolved fewer than half of the fixes, and executor choice significantly affected outcomes, highlighting the need for execution-based verification before such fixes reach users.

By Utku Boran Torun, Veli Karakaya, Eray T\"uz\"un
arXiv AI
Oct 9

Attention when you need

The paper investigates how a resource‑efficient agent should allocate attention when it incurs a metabolic cost. Using a normative signal‑detection model, it finds that optimal attention can follow either a ramping pattern—gradually increasing as evidence accumulates—or a rhythmic pattern, depending on task conditions and the cost of intense focus. Rhythmic attention emerges when belief updates are dominated by a stable temporal prior rather than new sensory evidence, and its frequency rises with higher rewards, shorter signals, and higher signal frequency.

By Grayson Matthew, Lokesh Boominathan, Yizhou Chen, Matthew McGinley, Xaq Pitkow
arXiv AI
Oct 9

Simultaneous Computation with Multiple Prioritizations in Multi-Agent Motion Planning

The paper proposes a method that allows agents in multi‑agent motion planning to compute with multiple prioritizations simultaneously, avoiding reliance on domain‑specific heuristics or costly iterative searches. This general approach is applied to receding‑horizon motion planning with system dynamics, and numerical experiments show near‑optimal prioritization and performance gains over state‑of‑the‑art methods with only a slight increase in computation time. Real‑time feasibility is demonstrated on a road network with ten vehicles in a Cyber‑Physical Mobility Lab.

By Patrick Scheffe, Julius Kahle, Bassam Alrifaee
arXiv AI
Oct 9

Agentic-TTT: Training test-time policy for test-time training

Agentic‑TTT introduces a test‑time policy that decides when and how to apply test‑time training (TTT) to large language models. By treating TTT procedures as tools and learning from the utility gains of its decisions, the policy can trade off performance improvements against computational cost. On a benchmark, Agentic‑TTT nearly doubles the utility of the base model, adapts to unseen domains, and demonstrates autonomous self‑improvement capabilities.

By Jiahao Lu, Mohan Kankanhalli
arXiv AI
Oct 9

A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization

The paper introduces AgenticBBO-Bench, a cross‑domain benchmark for evaluating large language model agents on black‑box optimization tasks such as synthetic functions, hyperparameter tuning, database tuning, chip design, and molecular design. Experiments show that agentic BBO outperforms direct LLM‑based methods in all five domains and beats the best numerical optimizers in four. The study also analyzes the impact of optimization tools, task semantics, prior knowledge, and the LLM’s role, finding that numerical tools are not consistently helpful, task semantics are broadly useful, and numerical optimizers can capitalize on agent‑generated search trajectories.

By Ming Chen, Rong-Xi Tan, Ke Xue, Yu-Jie Zhou, Taiye Lu, Zhi-Xuan Gao, Peng Xie, Zijun Shen, Chen Lu, Haopu Shang, Chao Qian
arXiv AI
Oct 9

Agentic Critical Training

Agentic Critical Training (ACT) is a reinforcement learning approach that trains language‑model agents to judge actions directly by pairing expert actions with alternatives and rewarding correct selection. Unlike imitation learning or self‑reflection, ACT does not require reference rationales and can be reused across model sizes. Experiments on Qwen3‑8B and Olmo‑3‑7B‑Instruct show that ACT improves performance on ALFWorld, WebShop, ScienceWorld, and outperforms supervised reflection baselines, with significant gains over IL and IL→RL baselines.

By Weize Liu, Minghui Liu, Sy-Tuyen Ho, Yongkyun Lee, Andrew Adams Schoen, Souradip Chakraborty, Xiyao Wang, Furong Huang
arXiv AI
Oct 9

SDPAD: A Fully Spike-Driven Pipeline for End-to-End Autonomous Driving

SDPAD is a fully spike‑driven end‑to‑end planning pipeline for autonomous driving that converts a pre‑trained ANN perception stack into integer‑spike form, lifts multi‑view images into a bird’s‑eye‑view space with a spike‑driven depth distribution, and plans via a spiking query transformer that fuses ego, agent, and map queries. On the nuScenes benchmark it matches strong ANN planners with an average L₂ error of 0.40 m and a collision rate of 0.12 %, while consuming only 69.9 mJ—less than 2 % of recent ANN baselines. In closed‑loop NAVSIM evaluation it achieves 86.3 PDMS, surpassing the previous SNN planner SAD by 4.3 points and matching mainstream ANN planners at a fraction of their energy.

By Chengjun Zhang, Yuhao Zhang, Jie Yang, Mohamad Sawan
arXiv AI
Oct 9

Runnable Commit Untangling for Coding Agents

The paper "Runnable Commit Untangling for Coding Agents" introduces RucTangle, an agentic method that untangles large, tangled patches from coding agents while ensuring each commit keeps the code runnable. It also presents TangleEval, an evaluation framework that measures how untangled commit histories aid agents in repairing bugs. Experiments on 131 agent-generated patches show RucTangle produces fully runnable histories, outperforming baselines that leave 20.6%-37.4% of commits unrunnable, and improves bug‑repair success by 5.2% absolute in pass@1.

By Jinfeng Jiang, Dongsun Kim, Dayi Lin, Zhou Yang
arXiv Computation and Language
Oct 9

GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness

GUI-KV is a plug‑and‑play key‑value cache compression method designed for graphical user interface agents that use vision‑language models. It exploits spatial saliency and temporal redundancy to prune cache entries, achieving near‑full‑cache accuracy while using modest memory budgets. Experiments on standard GUI agent benchmarks show significant reductions in decoding FLOPs and improvements in step accuracy compared to existing compression baselines.

By Kung-Hsiang Huang, Haoyi Qiu, Yutong Dai, Caiming Xiong, Chien-Sheng Wu
arXiv AI
Oct 9

Beyond Report Imitation: Clinically Aware Multi-Image Ultrasound Report Generation from Visible Evidence

The paper introduces CAMEO, a framework for generating ultrasound reports from multiple images that emphasizes clinical awareness and evidence grounding. It consists of three stages: learning visual-language primitives, cross-view evidence grounding using distilled image-report pairs, and clinically aware preference alignment with error-oriented pairs. Experiments on the USReport-Distilled benchmark show significant improvements over prior models in BLEU, ROUGE, METEOR, and ClinicalScore metrics.

By Yuchen Yang, Xin Wang, Lufan Wang, Yinghong Pan, Yujuan Feng, Yuqing Yang
arXiv Computer Vision
Oct 9

LR-V2X: Loss-Resilient Collaborative Perception under Low-Bandwidth Communication

LR‑V2X is a loss‑resilient collaborative perception framework for vehicular networks that reconstructs missing bird‑view (BEV) features from corrupted latent representations, even under severe packet loss. It converts corrupted latents into a spatial prior and uses ego‑vehicle context to recover BEV information, requiring only training under full‑communication conditions. Experiments on DAIR‑V2X and V2XREAL demonstrate that LR‑V2X maintains robust collaboration while reducing communication overhead by 64× compared to dense BEV fusion methods.

By Kang Yang, Tianci Bu, Peng Wang, Deying Li, Yongcai Wang
arXiv AI
Oct 9

Narrow and Deep: An Ontology Tower as the Knowledge of an LLM Agent for an Industrial Equipment System

The paper introduces the "ontology tower," a narrow-and-deep ontology designed for a single industrial equipment system. By enriching knowledge through derived quantities and operational lessons, the ontology improves large language model agents’ performance on plant tasks, raising error avoidance by ~20 percentage points and matching the largest models’ scores. In live safety tests, agents guided the system’s controlled variable into target ranges in 12 of 14 runs.

By Younghwan Joo, Sung-il Kim
arXiv AI
Oct 9

Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction

The paper introduces Synthesis Through Simulation (STS), a schema‑free data synthesis method that uses a large language model agent to generate enterprise data by interacting with policy‑enforcing APIs in simulated environments. STS ensures structural validity by construction and separates validity enforcement from distribution modeling. The Generalist Populator, an agent within STS, achieves high distributional fidelity (0.88 average marginal fidelity) and full constraint satisfaction across ten environments without needing database schemas, outperforming traditional statistical synthesizers and schema‑privileged agents.

By Yipeng Li, Ashutosh Hathidara, Jane Lo, Harshavardhan Abichandani, Gunraj Singh, Atin Ghosh
arXiv Machine Learning
Oct 9

CausalDreamer: Learning Predictive World Models with Latent Disentanglement

CausalDreamer enhances generative world models by freezing the original video tokenizer and re-encoding its latent space into a factored representation that separates controllable versus uncontrollable dynamics and reward-relevant versus reward-irrelevant information. The pretrained dynamics model is fine‑tuned to predict this factored representation, enabling more accurate model‑predictive planning. On the MMBench2 benchmark, CausalDreamer improves normalized scores by 14% on clean tasks and 25% on manipulated variants compared to the baseline, while both models perform poorly on entirely new environments.

By Prince Jha, Nils Lukas, Kun Zhang, Salem Lahlou
arXiv AI
Oct 9

MA-JEPA: Joint-Embedding World Models for Multi-Agent Reinforcement Learning

MA-JEPA is a stochastic world model designed for multi‑agent reinforcement learning that replaces observation reconstruction with joint‑embedding prediction. It uses a categorical latent state and a causal Transformer trained on posterior and action‑conditioned dynamics to generate imagined trajectories for actor‑critic learning. The model achieves strong performance on the SMAC benchmark, matching or surpassing the best reported mean win rates on half of the evaluated maps.

By Brandon Gary Kaplowitz, Osaze James Obahor, Christian Schroeder de Witt
arXiv AI
Oct 9

Agent-Controlled Forgetting for Tool-Using Agents: Reversible Context Curation in Practice

Agent-Controlled Forgetting for Tool-Using Agents: Reversible Context Curation in Practice explores a method where an agent selectively replaces detailed tool results with concise notes while archiving the originals for recovery. Using a Python harness, the approach reduces prompt tokens from 912,492 to 231,951 and cuts cumulative input tokens by 50%, yielding significant API cost savings. However, the method increases request count, slightly slows execution, and its effectiveness varies across different workloads.

By Jan-Peter Franke
arXiv AI
Oct 9

Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents

The paper introduces the Trajectory-Grounded Instruction Translator (TGIT), a front‑end that keeps an aerial vision‑and‑language navigation agent frozen while translating short, intent‑driven user instructions into executable commands. By prompting a language model with human‑written style examples, TGIT learns from the agent’s trajectory outcomes and improves success rates from 11.33 % to 37.93 % on weak inputs, and from 11.33 % to 32.51 % on real human instructions. The method also boosts performance on held‑out OpenFly data and provides recovery on CityNav and AirVLN benchmarks.

By Xi Chen, Zhe Liu, Xiaogang Xu, Jiafei Xu, Chunyi Zhou, Yuan Su, Rui Zeng, Tianyu Du, Kelu Yao, Chao Li, Shouling Ji