AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,733 stories · RSS feed

arXiv Computer Vision
Oct 23

SplatLab: Autonomous Scientific Discovery for 3DGS Research

SplatLab is an autonomous research system designed for 3D Gaussian Splatting (3DGS) that can reproduce published baselines, propose extensions, train models, and generate manuscripts with traceable results. It uses a verified-reproduction backbone, a cascade-weighted Elo tournament for candidate evaluation, a standing falsifier for stress‑test survival, and a seven‑category 3DGS taxonomy to ensure reproducibility and transferability. In PSNR‑targeted experiments, SplatLab’s manuscripts outperform their reproduced baselines by up to +3.78 dB, compressing the research cycle from months to days of autonomous compute.

By Seemandhar Jain, Manmohan Chandraker
arXiv Computer Vision
Oct 23

Unified Trajectory and 3D Pose Forecasting for Multi-Agent Sports Analysis

The paper introduces a unified model for forecasting both global trajectories and 3D body poses of multiple agents in sports settings. It uses a socio‑temporal encoder that fuses trajectory and pose inputs, shared factorized attention, per‑joint loss reweighting, and a frequency‑domain smoothing term to improve predictions. The authors also correct evaluation biases by adopting a non‑overlapping protocol and demonstrate state‑of‑the‑art results on WorldPose and 3DPW datasets, especially in pose accuracy.

By Marc Peral, Guillem Capellera, Luis Ferraz, Antonio Rubio, Stella Gra{\ss}hof, Dan Witzner Hansen, Antonio Agudo
arXiv Computer Vision
Oct 23

Asking Less, Adapting Better: Feedback-Efficient Test-Time Adaptation for Vision-and-Language Navigation

The paper introduces FETA, a Feedback‑Efficient Test‑Time Adaptation method for Vision‑and‑Language Navigation. FETA uses a hierarchical sample selection strategy to identify informative trajectories and critical actions, querying only a single binary feedback per action. The method updates the policy at the queried step with a discrepancy‑aware entropy objective, achieving state‑of‑the‑art performance while reducing human interventions and maintaining robustness to noisy feedback.

By Zixu Wang, Huaquan Zhang, Shoukai Xu, Jinwu Hu, Zhiquan Wen, Mingkui Tan
arXiv Computer Vision
Oct 23

PhyEvolve: Multi-Agent State-Evolution Planning for Physics-Aware Video Generation

PhyEvolve is a multi‑agent framework that improves physics‑aware video generation by enhancing natural‑language prompts through structured state‑evolution planning. It employs three agents—a Director, a Physical Evolver, and a Composer—to ground scenes, identify physical mechanisms, and construct causally linked state transitions that guide existing generators without retraining. Experiments on four major benchmarks show that PhyEvolve outperforms the prior agentic baseline by an average of 7.9 percentage points, demonstrating that explicit causal planning boosts physical consistency and state continuity in generated videos.

By Yang-Hao Zhou, Lixing Tan, Zhiqi Li, Guang Yang, Xiao Yang
arXiv Computer Vision
Oct 23

Reinforcing intra-operative tumour localisation with weak supervision from pre-operative appearances

The paper introduces a method that uses weak supervision from pre‑operative magnetic resonance (MR) images to improve intra‑operative tumour localisation on computed tomography (CT) scans. A classifier trained on MR data assigns tumour‑presence probabilities to CT image windows, and these signals guide a reinforcement learning agent to locate tumours. Experiments on an external hepatocellular carcinoma cohort show that a synthetic transfer approach—converting CT to synthetic MR before localisation—outperforms direct transfer, few‑shot adaptation, and CT‑only baselines.

By Sidhart Sami, Ali Sharjeel, Sri Atluri, Miao Zhao, Caroline Roney, Eirini Marouli, Shaheer U. Saeed
arXiv Computer Vision
Oct 23

Hybrid-PCGC: Hybrid Point Cloud Geometry Compression Bridging Generalization-based Model and Implicit Neural Representation

Hybrid-PCGC is a point cloud geometry compression method that blends learned generalization with implicit neural representations. It treats compression as a joint geometry-and-model bitrate allocation problem, optimizing instance‑specific weight offsets as transmitted variables. The approach separates dataset‑agnostic prior modeling from instance‑specific adaptation using a Frozen General Module, a Dual Trainable Adaptive Module, an Adaptive Scope Controller, and a Supervised Offset Compression Module, achieving significant bitrate reductions over existing methods.

By Wenjie Huang, Qi Yang, Shuting Xia, He Huang, Zhu Li, Yiling Xu
arXiv AI
Oct 23

Self-Improving Agents with World Model Predictive Paths

The paper introduces a test‑time framework that blends Model Predictive Path Integral (MPPI) control with a dual‑branch experience memory for large language model agents. At each decision step, the agent generates multiple candidate actions, uses a world model to simulate their outcomes, and evaluates these predictions for task progress and risk to select the best action. The dual‑branch memory allows the planner to refine action generation and the world model to improve future observation predictions, leading to strong performance on ALFWorld and ScienceWorld benchmarks.

By Haori Lu, Tianyu Gao, Yaoyao Liu, Yingying Li
arXiv AI
Oct 23

An Evidence-Driven Multi-Agent Compound AI System for Head and Neck Proton Radiotherapy Planning

The paper presents an evidence-driven, multi-agent AI system for head and neck proton radiotherapy planning. It combines a DRL-based planning agent that generates multiple treatment plans with an LLM-based conversational agent that refines the chosen plan through natural language interactions. The system, trained on 21 patients, improves instruction-following and domain-specific reasoning, yielding more clinically appropriate trade-offs in patient-specific plan refinement.

By Qingqing Wang, Liqiang Xiao, Chang Chang
arXiv Computation and Language
Oct 23

Social Life Simulation for Non-Cognitive Skills Learning

The paper introduces Simulife++, an interactive platform powered by a large language model that lets users create stories with AI characters in social scenarios. It expands human-AI interaction to include a Sage Agent, a bystander AI that offers guidance and perspectives to promote reflection on non‑cognitive skills. A within‑subject study shows that the Sage Agent significantly boosts users’ reflection on motivation, self‑perceptions, resilience, and narrative transportation, while also enhancing social communication and decision‑making.

By Zihan Yan, Yaohong Xiang, Yun Huang
arXiv Computation and Language
Oct 23

Efficient Best-Of-N with Radial Consensus Score

The paper introduces Radial Consensus Score (RCS), a training‑free method for selecting the best answer from a set of candidate responses generated by large language models. RCS computes a weighted Fréchet mean of answer embeddings to form a semantic center and ranks candidates by their radial distance to this center, thereby capturing semantic consistency rather than surface agreement. Experiments on five open‑weight models and multiple benchmarks show that RCS matches or surpasses strong baselines across all sampling budgets, especially when correct answers are lexically fragmented, and can replace majority voting in multi‑agent debate and black‑box settings.

By Manh Nguyen, Sunil Gupta, Hung Le
arXiv Computation and Language
Oct 23

IfcLLM: Natural Language Querying of IFC Models through Complementary Relational and Graph Representations

IfcLLM is a framework that enables natural language querying of Industry Foundation Classes (IFC) models by combining relational and graph data representations. It routes attribute retrieval queries to a relational backend and spatial reasoning queries to a graph backend, with an LLM agent coordinating the process through iterative retry-and-refine reasoning. In tests on 30 query scenarios across three IFC models, the best configuration achieved 96% accuracy, and experiments with open-weight LLMs suggest the approach can be deployed locally in data-sensitive AEC environments.

By Rabindra Lamsal, Sisi Zlatanova, Haowen Xu, Yafei Sun, Xuesong Shen
arXiv Computer Vision
Oct 23

FilmFlow: Agentic System for 3D Virtual Film Cinematography

FilmFlow is an agentic framework that transforms natural‑language instructions or reference videos into explicit camera trajectories for dynamic 3D environments. It uses a Director agent that composes high‑level motions from concurrent atomic operations, supported by Interpreter, Scout, and Critic agents for intent extraction, viewpoint selection, and iterative refinement. The authors also introduce CinemaBench, a benchmark of 150 Blender scenes with 750 text‑directed tasks and 60 video‑reference tasks, where FilmFlow outperforms baselines and receives the highest user preference.

By Yiming Zhang, Jiarui Guo, Haojia Wei, Yifei Liu, Yifan Yang, Hongjie Zhang, Xue Yang, Zhihang Zhong
arXiv Computer Vision
Oct 23

AA-ViT: Anatomically Aware Vision Transformer with Structural and Frequency Guidance for Contrast Enhanced Brain MRI Synthesis

The paper introduces AA‑ViT, an anatomically aware vision transformer that synthesizes contrast‑enhanced brain MRI from pre‑contrast modalities (T1, T2, FLAIR) by incorporating structural and frequency guidance. Experiments on the BraTS 2021 dataset show that AA‑ViT preserves anatomical and lesion boundaries, achieving higher PSNR and SSIM than existing methods. A clinical evaluation by four experts on 19 glioma cases yielded a mean score of 3.94/5, indicating promising clinical relevance.

By Talha Meraj, Tom Flannery, Charlie Cummins, Matt Townend, Thomas C Booth, Peter Crossley, Michael McCann, Ian Overton, Saritha Unnikrishnan
arXiv AI
Oct 23

Correlated Failures of AI Approval Monitors

The paper investigates how AI approval monitors can fail together when they share a base model or poisoned input, using concepts from distributed computing to identify vulnerable monitor sets. Experiments on twelve open‑weight monitors and several closed models show that fooling a monitor into approving fraud is limited to its own evidence source, while blocking correct payments requires overcoming injection resistance. A rule that counts distinct evidence sources rather than votes reduces false rejections, and adding a rule engine that cannot be persuaded restores safety and liveness against the measured attacks.

By Luca Zanolini
arXiv AI
Oct 23

Automated Materials Data and Knowledge Extraction using Machine Learning and Large Language Models

The review traces the development of machine‑learning techniques for extracting materials data and knowledge from scientific literature, progressing from rule‑based pipelines to NLP models, transformer encoders, and finally large language models and agentic systems that handle text, tables, and figures. It highlights how these methods enable the creation of queryable materials databases, knowledge graphs, generic question answering, AI co‑scientists, chatbots, and closed‑loop discovery with autonomous labs. The paper critically evaluates datasets, extraction accuracy, computational costs, and offers practical guidelines, while outlining open challenges such as full tetrahedron extraction, benchmark scarcity, and terminology resolution.

By Aravindan Kamatchi Sundaram, B. Pabitramohan Prusty, Rohit Batra
arXiv AI
Oct 23

RocketAgent: A Long-Horizon Engineering Agent for Multidisciplinary Design of Liquid-Rocket Thrust Chambers

RocketAgent is a long‑horizon engineering agent designed to streamline the multidisciplinary preliminary design of liquid‑rocket thrust chambers. It employs a single plan‑owning Coding Agent that coordinates performance sizing, subsystem optimization, geometry generation, and multiphysics assessment, while a provenance‑aware knowledge graph and a typed Design Intermediate Representation maintain consistent design information and enforce revision‑aware checks. In a representative simulation‑based design, RocketAgent successfully navigated infeasible cooling searches, incorporated engineer‑approved operating‑point revisions, and coordinated subsequent geometry generation and multiphysics assessment to support final configuration selection, with controlled tests confirming the proper invalidation of superseded inputs.

By Junxiang He, Runze Mao, Kun He, Teng Zhang, Liming Zheng, Ke Xiao, Zhi X. Chen
arXiv Machine Learning
Oct 23

Learning to Drive in New Cities Without Human Demonstrations

The paper presents NOMAD, a method that uses self‑play multi‑agent reinforcement learning to adapt autonomous driving policies to new cities without any human demonstration data from those cities. By leveraging only the target city’s map and meta‑information, NOMAD improves task success rates and trajectory realism in simulation, offering a scalable alternative to data‑intensive city‑transfer approaches. The approach demonstrates that reliable driving can be achieved in substantially different urban environments through map‑based self‑play.

By Zilin Wang, Saeed Rahmani, Daphne Cornelisse, Bidipta Sarkar, Alexander David Goldie, Jakob Nicolaus Foerster, Shimon Whiteson
arXiv Machine Learning
Oct 23

Markovian Generation Chains in Large Language Models

The paper introduces Markovian generation chains, an iterative inference process in large language models where each step uses a prompt template and the previous output as input, excluding prior data. Experiments in iterative rephrasing and round‑trip translation show that outputs either converge to a small recurrent set or continue generating novel sentences within a finite horizon. Sentence‑level Markov chain modeling reveals that the process can either increase or reduce diversity, depending on factors such as temperature and the initial input sentence.

By Mingmeng Geng, Amr Mohamed, Guokan Shang, Michalis Vazirgiannis, Thierry Poibeau
arXiv Machine Learning
Oct 23

Projecting Latent RL Actions: Towards Generalizable and Scalable Graph Combinatorial Optimization

The paper introduces projection agents for graph combinatorial optimization, leveraging the Wolpertinger framework and a shared graph neural network latent space to predict latent actions in a single forward pass. This approach achieves up to 6.41× faster inference and 75% better generalization across diverse benchmarks compared to existing methods, while enabling strong reinforcement learning performance in large decision spaces. The authors also release the LaGCO‑RL Python library to facilitate reproducibility and adaptation to new GCO benchmarks.

By Franco Terranova (UL, LORIA, RESIST), Guillermo Bernardez (UC Santa Barbara - UCSB), Albert Cabellos-Aparicio (UPC), Nina Miolane (UC Santa Barbara - UCSB), Abdelkader Lahmadi (LORIA, UL, RESIST)