AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,750 stories · RSS feed

arXiv Computer Vision
Sep 16

Exploring 2D backbone effects for indoor semantic occupancy prediction

The paper investigates how different 2D image backbones affect indoor semantic occupancy prediction in RGB‑D pipelines. By keeping the projection, depth branch, and occupancy head constant and swapping only the image encoder, the authors find that stronger backbones such as DINOv2 and BLIP2 significantly raise mIoU scores compared to the default ResNet‑50. These results show that the choice of image backbone is a major determinant of 3D occupancy accuracy, outweighing many specialized 3D modules.

By Shizhang Fanga, Wanling Yea, Qi Zheng
arXiv Computation and Language
Sep 16

Enhancing Accessibility of Medical Texts through Large Language Model-Driven Plain Language Adaptation

The paper explores how Large Language Models (LLMs) such as GPT-4o-mini, Gemini-1.5-pro, and LLaMA can be used to automatically simplify complex medical texts into plain language, a process known as Plain Language Adaptation (PLA). It evaluates different prompting strategies, fine‑tunes models with QLoRA, and incorporates Mixture‑of‑Agents (MoA) techniques to improve adaptability and robustness. The results show that LLM‑driven PLA effectively makes healthcare information more comprehensible while preserving essential content.

By Ting-Wei Chang, Hen-Hsen Huang, Hsin-Hsi Chen
arXiv Machine Learning
Sep 16

EMODY Flow: Emotion-Aware Audio-Driven Full-Body Motion Generation

EMODY Flow is a lightweight flow‑matching framework that generates synchronized full‑body motion and facial expressions conditioned on speech and emotion. It attaches to a frozen Qwen‑3 Omni model, reusing its audio codecs to drive two parallel DiT generators for SMPL‑X body pose and FLAME facial expressions. An auxiliary emotion classifier at training time restores emotion sensitivity, enabling EMODY Flow to achieve state‑of‑the‑art gesture quality on BEAT2 and zero‑shot facial animation on TFHP, with significant improvements in FGD, Beat Correlation, and Diversity metrics.

By Harsh Kumar Agarwal, Xavier Alameda-Pineda, Olivier Perrotin
arXiv AI
Sep 16

QART: A Quantum-Classical Hybrid Architecture for Long-Horizon Reasoning -- Exploring a Conditional Path toward Quantum Scaling

QART is a quantum‑classical hybrid architecture that augments a language model with quantum encoding, CIM‑based QUBO optimization, and quantum decoding to improve long‑horizon reasoning. The authors claim that, under certain assumptions, QART can maintain a non‑zero probability of recovering an optimal reasoning path while traditional autoregressive LLMs see their acceptance probability drop to zero as cumulative risk grows. Experiments on six benchmarks with three backbone models show that QART outperforms the baselines in 14 of 15 pairings, with relative gains up to 84.0% on SciCode.

By Lehao Lin, Yuheng Cheng, Guolong Liu, Yao Li, Xuning Tan, Xiyuan Zhou, Ruixi Zou, Shi Wang, Huan Zhao, Wenxuan Liu, Haifeng Wu, Junhua Zhao
arXiv AI
Sep 16

Mapping U.S. Federal AI Governance Against Sector Vulnerability

The study evaluates 684 U.S. federal AI governance documents for how they address 14 sectors and 24 AI risks, measuring both breadth and depth of coverage. It finds that risks related to robustness, system security, and governance are more frequently and substantively discussed than socioeconomic, environmental, and emerging risks, and that sectors such as public administration, national security, information, and scientific services receive higher coverage than finance and healthcare. By comparing these coverage patterns with expert vulnerability assessments, the authors identify potential gaps in AI governance that could inform future policy and industry decisions.

By Ho Ting Hung, Angelica Chowdhury, James Teague, Simon Mylius, Spencer Michaels, Peter Slattery, Alexander Saeri, Neil Thompson
arXiv AI
Sep 16

A Cyber Range Evaluation of Autonomous Network Incident Response Agents

The study evaluates autonomous agents that respond to network intrusions within a cyber range designed for human operator training. Using an emulated network with variable topology, red‑team attacks, and simulated users, the agents aim to block unauthorized access while minimizing defensive costs. Experiments compare heuristic policies with reinforcement‑learning‑derived policies, finding that the latter generally defend more efficiently, though performance varies with adversary strategy and user simulation.

By Jakob Nyberg, Teodor Sommestad, Andrei Buhaiu, Joakim Loxdal, Pontus Johnson, Mathias Ekstedt
arXiv Computation and Language
Sep 16

Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act

The paper investigates how reinforcement learning can cause large language model agents to adopt shortcut policies for tool use, relying on superficial prompt cues rather than actual task needs. By creating synthetic environments that mix factual QA and math reasoning, the authors show that agents often invoke tools when cues are present, even when those tools are unnecessary, with spurious invocation rates rising up to 39%. They find that shortcut learning occurs mainly when agents have already mastered the target tool and that semantic alignment between cues and tools amplifies the effect. To counter this, they propose a dense, decision-level reward where an LLM judge assesses tool necessity, which reduces cue-driven tool use while maintaining performance.

By Yiwei Yang, Haoxiang Zhang, Bingbing Wen, Yao Lu, Yuchen Wu, Lei Zhang, Julian McAuley, Pan Lu, Bill Howe
arXiv AI
Sep 16

Multi-Agent Collaboration for Automated Design Exploration on High Performance Computing Systems

The paper introduces MADA, a Large Language Model–powered multi‑agent framework that coordinates specialized agents—Job Management, Geometry, and Inverse Design—to automate complex design workflows on high‑performance computing systems. In the context of Richtmyer–Meshkov Instability suppression for Inertial Confinement Fusion, MADA iteratively refines designs by launching ensemble simulations, generating meshes, and proposing new designs based on simulation outcomes, achieving improved suppression with minimal manual effort. The framework demonstrates how coordinated reasoning, simulation, and specialized tools can be scaled for rapid, automated design exploration.

By Harshitha Menon, Charles F. Jekel, Kevin Korner, M. Giselle Fernandez-Godino, Brian Gunnarson, Nathan K. Brown, Michael Stees, Walter Nissen, Meir H. Shachar, Dane M. Sterbentz, William J. Schill, Yue Hao, Robert Rieben, William Quadros, Steve Owen, Scott Mitchell, Ismael D. Boureima, Jonathan L. Belof
arXiv AI
Sep 16

KaiNinja: Extending Native 3D Generators to the Part Level

KaiNinja extends the native 3D generator TRELLIS.2 to produce part-level meshes by introducing a dual‑volume representation that overcomes the single‑sheet limitation of the O‑Voxel grid. It maintains TRELLIS.2’s speed and quality while eliminating the need for external segmentation, and is trained on diverse data including CAD models and assets created by an LLM‑driven agent. The method improves whole‑object fidelity and outperforms other part‑generation pipelines, reducing Chamfer distance by 40% and increasing strict part F‑score by 16%.

By Ruihan Yu, Lian Fu, Muyao Niu, Zheng-hui Huang, Yu-Ju Tsai, Sho Kuno, Fengbo Lan, Yonghao Yu, Erwin Wu, Ming-Hsuan Yang, Kaipeng Zhang, Zhixiang Wang
arXiv Machine Learning
Sep 16

Multi-Agent Learning with Cooperation-Driven Optimization Dynamics

The paper proposes a cooperation mechanism for multiple small neural network agents that share predictions during training to reduce model complexity while maintaining performance. By incorporating shared predictions into the loss function, agents influence each other's weight updates through strategies such as voter, majority, and weighted average models. Experiments on standard benchmarks show that several small agents can outperform a single large model, achieving comparable accuracy with fewer parameters and lower computational cost.

By Jarod Ketcha Kouakep, Sreyvi UANN, Timoteo Carletti
arXiv Machine Learning
Sep 16

Neural Operator Learning for Collision-Aware Trajectory Planning of Spacecraft Swarms

The paper presents a permutation‑equivariant neural operator that learns to generate collision‑free, fuel‑efficient trajectories for spacecraft swarms by mapping distributions of initial and target states, as well as obstacle states, to trajectory outputs. The operator is self‑supervised and, when paired with a batched Gauss‑Newton step, enforces exact orbital dynamics and further reduces fuel consumption. Trained on ten spacecraft, the model generalizes zero‑shot to swarms of 1,000 spacecraft and 11,000 obstacles, achieving accuracy comparable to a per‑agent optimal control solver while maintaining collision avoidance.

By Sidhdharth D. Sikka, Suyi Gao, Zehui Lu, Rongjie Lai, Shaoshuai Mou
arXiv AI
Sep 16

Turn-level Multiscale Density Ratio Estimation for LLM Agents

The paper introduces Turn-level Multiscale Density Ratio Estimation (tlm‑DRE), a post‑training technique for large language model agents that assigns varying weights to different turns and employs asymmetric token‑level training based on positive‑negative gaps across multiple turns. Unlike conventional alignment methods such as PPO, DPO, DIL, and GRPO, which mainly target single‑turn tasks, tlm‑DRE is designed for complex multi‑turn reasoning. Experiments on diverse agent benchmarks demonstrate that tlm‑DRE competes with traditional alignment methods and enables LLMs to perform robustly in both in‑domain and out‑of‑domain multi‑turn tasks.

By Zishuo Zhao (Alibaba Group), Kai Chen (Alibaba Group), Ao Li (Alibaba Group), Yuan Liu (Alibaba Group)
arXiv Computation and Language
Sep 16

EviSI: An Evidence-Based Evaluation Agent for Simultaneous Interpreting

EviSI is an evidence‑based evaluation agent for low‑latency simultaneous speech‑to‑speech translation. It combines Multidimensional Quality Metrics with interpreter‑developed criteria, using shared source evidence to assess four dimensions—Anchor, Event, Logic, and Fluency—while deduplicating verified errors before scoring. On English‑to‑Chinese and Chinese‑to‑English data, EviSI’s rankings correlate strongly with human judgments, outperforming BLEU and COMET, and its multilingual extension maintains these correlations across five language directions.

By Ben Yan, Zongyao Li, Xiaoyu Chen, Daimeng Wei, Weidong Liu, Huan Zhao, Chong Li, Yaode Wang, Yuzhe Shang
arXiv Computer Vision
Sep 16

World-Action Models for Robot Learning and Control: A Survey

The survey "World-Action Models for Robot Learning and Control" reviews recent advances in coupling future world prediction with executable action generation for robots in open environments. It clarifies the scope of World-Action Models (WAMs) relative to conventional world models, model-based RL, and Vision‑Language‑Action policies, and organizes existing methods through a unified taxonomy covering representations, transition modeling, action interfaces, architectures, training pipelines, data modalities, and scaling strategies. The paper also surveys applications in manipulation, navigation, and autonomous driving, summarizes datasets, benchmarks, and metrics, and discusses key challenges such as action alignment, spatial consistency, long‑horizon memory, and efficient inference.

By Zuxing Lu, Hongjia Zhai, Guanzhi Wang, Huajian Zeng, Jiaqi Yang, Jingyu Liu, Lei Cheng, Yuantai Zhang, Yuheng Qiu, Zezhou Cheng, Ivan Laptev, Danfei Xu, Benjamin Riviere, Giuseppe Loianno, Eric Xing, Xingxing Zuo