AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,805 stories · RSS feed

arXiv Computer Vision
Sep 1

Neural 3D Object Reconstruction with Small-Scale Unmanned Aerial Vehicles

arXiv:2509.12458v3 Announce Type: replace-cross Abstract: Miniaturized Uncrewed Aerial Vehicles (UAVs) can access indoor and hard-to-reach spaces, but severe constraints on payload and autonomy have...

By \`Almos Veres-Vit\`alyos, Filip Lemic, Daniel Johannes Bugelnig, Joan Bernaus Casades\'us, Genis Castillo Gomez-Raya, Sergi Abadal, Bernhard Rinner, Xavier Costa-P\'erez
arXiv Computer Vision
Sep 1

MANTLE: A Framework for Adaptive In-Situ Planetary Perception Using a Modular Uplink Principle

MANTLE is a multi‑task adaptive network designed for planetary perception, featuring a shared DINOv2 backbone with separate heads for landform classification and boulder segmentation. Trained on HiRISE and MSL imagery, it achieved 92.56% accuracy on seven Martian terrain classes and a 0.753 IoU for boulder segmentation, with strong cross‑sol generalization. The framework follows the Modular Uplink Principle, allowing lightweight task‑specific heads to be trained on Earth and uplinked to the rover without retraining the core model.

By Pranav Durai, Gary Doran
arXiv Computer Vision
Sep 1

Can Retrieval Heads See Images? Multimodal Retrieval Heads in Long-Context Vision-Language Models

arXiv:2605.27243v3 Announce Type: replace Abstract: Large vision-language models increasingly rely on long-context modeling to reason over documents, hour-level videos, and long-horizon agent traject...

By Aaron Branson Cigres Li, Zhaowei Wang, Yu Zhao, Yiming Du, Haobo Li, Xiyu Ren, Ginny Wong, Simon See, Lishu Luo, Haodong Duan, Pasquale Minervini, Yangqiu Song
arXiv Machine Learning
Sep 1

AdaptAV: Continuous Adaption of Vision Models for Autonomous Vehicles Using Cloud-based Oracle

The paper introduces AdaptAV, a system that continuously adapts vision models for autonomous vehicles by retraining them on the cloud using data uploaded from the vehicles. It leverages powerful cloud compute resources and a highly accurate oracle model to guide the retraining process, producing a new model that is then transmitted back to the vehicle. This approach aims to improve inference accuracy over time while maintaining the fast inference speeds required for on‑vehicle deployment.

By Yuheng Zhu, Dhruva Ungrupulithaya, Boluo Ge, Man-Ki Yoon
arXiv Machine Learning
Sep 1

SpIDER: Spatially Informed Dense Embedding Retrieval for Software Issue Localization

SpIDER is a dense retrieval method that combines LLM reasoning with graph-based exploration of codebases to locate relevant functions, classes, or files for user queries. It introduces a graph-structured benchmark, SpIDER-Bench, covering multiple programming languages and demonstrates significant recall improvements over traditional BM25 and dense approaches. The method’s graph-based candidate expansion provides auditable structural reasons for each retrieved item while keeping the retrieval budget fixed.

By Shravan Chaudhari, Rahul Thomas Jacob, Jiajun Cao, Shihab Rashid, Mononito Goswami, Christian Bock
arXiv Computation and Language
Sep 1

Identifying and Mitigating Bottlenecks in Role-Playing Agents: A Systematic Study of Disentangling Character Profile Axes

The paper introduces a diagnostic framework to disentangle the effects of character profile axes—Familiarity, Structure, and Disposition—on large language model role‑playing agents. Experiments on 211 personas and five LLMs show that Familiarity and Structure have little impact, whereas Disposition, particularly immoral traits, consistently degrades performance. The authors propose Field‑Aware Contrastive Decoding (FACD), a training‑free method that mitigates this performance gap without harming moral‑character performance.

By Yonghyun Jun, Junhyuk Choi, Jeonghyun Park, Jihyeong Park, Liu Nicole Geumheon, Hwanhee Lee
arXiv Computation and Language
Sep 1

GPAgentBench-2K: Benchmarking Large Language Model Agents in Complex Clinical Action Space

GPAgentBench-2K is a new benchmark that evaluates large language model agents in primary‑care clinical decision‑making using a constrained Markov Decision Process (CMDP). It is built from expert‑validated records of real GP encounters and includes six foundational clinical actions, a workflow prior, and safety‑informed abstention as an outcome. Testing 16 state‑of‑the‑art LLMs shows that performance drops as the action space grows and that even the best models violate safety constraints in more than half of high‑risk cases, highlighting a gap between clinical quality and safety.

By Boqi Chen, Xudong Liu, Yunke Ao, Heejin Do, Jianing Qiu
arXiv Computation and Language
Sep 1

PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback

PaperBanana-Interact is a multi-agent system designed to refine scientific diagrams through multi-turn human feedback. The authors introduce MTPaperBananaBench, a benchmark with 292 images and 3,518 user requirements, and a user simulator that generates natural language feedback at each turn. Experiments show that PaperBanana-Interact consistently improves diagram quality, outperforming baseline systems by 11.9–18.6 points and reducing forgetting by 3.7–6.2 points.

By Xueqing Wu, Ashwin Balasubramanian, Bingxuan Li, Dawei Zhu, Kai-Wei Chang, Yale Song, Yiwen Song, Rui Meng, Tomas Pfister, Nanyun Peng
arXiv AI
Sep 1

Learning Simple Test-Time Environments for LLM Web Agents

The paper introduces Test-Time Environment Decomposition (TTED), a label‑free learning method that allows large language model agents to break down complex web environment observations into simpler sub‑modules during inference. By learning from experience within these sub‑environments, agents can compose the gained knowledge to improve performance in the full environment. Experiments on synthetic and realistic benchmarks show that this approach enhances compositional generalization and boosts real‑world web automation tasks.

By Junxuan Li, Zijun Liu, Ziyi Huang, Peng Li, Yuzhou Liu, Ming Yan, Yang Liu
arXiv Computer Vision
Sep 1

Projection-Aware End-to-End Learned Video Compression for 360-Degree Video

The study examines how different two‑dimensional projections of spherical 360‑degree video affect end‑to‑end neural compression. Seven JVET‑360Lib formats were evaluated using a scale‑space flow model on standard test sequences, with performance measured by PSNR, spherical PSNR, weighted spherical PSNR, and BD‑rate. Results show that equirectangular and padded equirectangular projections yield the best compression efficiency with the neural model, while cubemap‑based formats excel with conventional HM‑16.16 codecs, highlighting that projection choice is codec‑dependent.

By Niloofar Maani
arXiv Computation and Language
Sep 1

Evaluating the Capabilities of LLMs for Persuasive Dialogue

The paper introduces “Persuasio”, a multi‑agent dialogue platform that uses a formal argumentation theory to adjudicate winners in free‑text debates. Using this system, the authors generated 192 debates on a UK political topic involving humans and large language models (LLMs), and evaluated 22 interlocutors through automated adjudication and 9,702 crowdsourced pairwise judgments across 1,386 annotation instances. The results show a consistent decoupling between subjective persuasiveness—where LLMs dominate—and formal argumentative strength—where humans remain competitive, with multi‑agent and retrieval‑augmented variants widening this gap.

By Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn
arXiv Computer Vision
Sep 1

Weaving Visual Narratives: Agentic Image Bundle Composition Beyond Atomic Visual Matching

The paper introduces Image Bundle Composition (IBC), a new paradigm that moves beyond point-wise image matching to dynamically assemble cohesive image bundles from large, unstructured photo collections. It presents IBCBench, a benchmark with over 109,000 images and 667 verified queries, and proposes BundleWeaver, an agentic framework that uses a Large Language Model for relational role discovery and a Vision‑Language Model for bundle verification. Experiments show that BundleWeaver outperforms existing embedding and decompose‑and‑rerank methods, underscoring the importance of relational composition over atomic scoring.

By Rong Shan, Tianyi Xu, Congmin Zheng, Wenteng Chen, Jiachen Zhu, Junjie Wu, Teng Wang, Weiwen Liu, Changwang Zhang, Weinan Zhang, Jun Wang, Jianghao Lin
arXiv AI
Sep 1

KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs

The paper introduces KVoiceBench, KOpenAudioBench, and KMMAU—three Korean speech benchmarks created through agent-driven frameworks that adapt existing SpokenQA and ASR resources into Korean SpokenQA and audio understanding tasks. These benchmarks total 12,345 samples and are publicly released to evaluate SpeechLMs beyond English. The authors benchmark eight recent SpeechLMs, revealing significant English‑Korean performance gaps and divergent rankings between SpokenQA and audio understanding, highlighting multilingual weaknesses not apparent in English-only tests.

By Haechan Kim, Seungjun Chung, Inkyu Park, Jihoo Lee, Jonghyun Lee
arXiv AI
Sep 1

Free Speech and Artificial Intelligence

The paper examines how artificial intelligence impacts freedom of expression, focusing on two technologies: social media recommendation algorithms and conversational AI chatbots. It argues that recommendation algorithms shape speech visibility in the digital public sphere, making them relevant to free‑speech philosophy, while conversational AI raises questions about granting speech rights to agents and users’ right to information. The work also explores corporate speech rights for platforms and developers, aiming to highlight unresolved issues from AI’s rapid growth.

By Etienne Brown
arXiv Computation and Language
Sep 1

LLMs Can't Play Hangman: On the Necessity of a Private Working Memory for Language Agents

The paper argues that large language models (LLMs) lack a private working memory, which is essential for tasks that require hidden state, such as interactive games. It introduces Private State Interactive Tasks (PSITs) and proves that chat-based LLMs cannot maintain a secret state while producing consistent public responses. Empirical tests confirm that standard LLMs and retrieval-based baselines fail, while a new architecture with explicit private memory succeeds.

By Davide Baldelli, Ali Parviz, Amal Zouaq, Sarath Chandar
arXiv Computation and Language
Sep 1

Aspire: Can Models Self-Evolve from Vague Goals?

The paper introduces ASPIRE, a benchmark that challenges language model agents to self‑evolve from vague, natural‑language goals without explicit evaluation metrics. In ASPIRE, agents must interpret the goal, select data and update strategies, and decide when to evaluate, all while the downstream tasks remain hidden. Experiments show that while agents can complete training loops, weight‑level improvements are sparse and unstable, and the best evolved harness still falls short of a strong engineered baseline.

By Yuhao Wu, Jingyuan Zhang, Jiajun Shi, Yuxuan Zhang, Xinping Lei, Junting Zhou, Zexuan Wang, Yuchen Wu, Huan Zhou, Duo Wang, Yinzhu Piao, Yongchang Peng, Yunfeng Shi, Jin Chen, Zuo Wang, Jinkai Liu, Jiaheng Liu, Wenxuan Zhang, Shen Yan, Wenhao Huang, Ge Zhang
arXiv AI
Sep 1

Motus2: A Self-Evolving General World Model for Dexterous Manipulation

Motus2 is a self‑evolving general world model designed for dexterous manipulation. It integrates a shared‑weight model that offers three control interfaces—a policy, a simulator, and an evaluator—forming a closed decision‑and‑learning loop for policy improvement. The system scales both model size and data, progressing from large‑scale monocular egocentric data to synchronized stereo data and robot‑domain adaptation, while also incorporating tactile feedback and a biomimetic platform with dual arms and hands.

By Hongzhe Bi, Zihao Zhou, Yihang Tang, Jingrui Pang, Shuhe Huang, Haitian Liu, Runqing Wang, Shuai Huang, Yichen Wang, Yiming Cheng, Ruowen Zhao, Zhenghua Li, Hengkai Tan, Xiaolong Liu, Jinhui Wan, Jiabao Liu, Min Zhao, Fan Bao, Jun Zhu
arXiv AI
Sep 1

Forward-Deployed Full-Stack Engineering for Autonomous Cloud MLOps

The paper introduces a multi‑agent framework that transforms natural‑language MLOps tasks into verified repositories and operational cloud deployments. It uses a stateful Graph Orchestrator to coordinate agents for repository generation, review, execution, verification, release, and monitoring, ensuring lifecycle transitions only occur when supported by verifiable evidence. The framework, implemented on Google Cloud Platform, demonstrates prevention of unsupported transitions and drives each run toward a verified deployment or an auditable failure.

By Sagar Srinivas Sakhinana, Venkataramana Runkana
arXiv AI
Sep 1

LOCI: A Locator-Critic with Refinement Loop

LOC I (Locator‑Critic) is a training‑free framework that separates visual search from evidence verification in Vision‑Language Models. It uses a Locator agent to propose candidate visual evidence and a Critic agent to assess its relevance, engaging in an iterative refinement loop that progressively improves the evidence until it is sufficient to answer a question. The approach yields state‑of‑the‑art results on several complex visual benchmarks, boosting accuracy for both open‑weight models like Qwen3‑VL and proprietary models such as Gemini 2.5 Pro.

By Walid Bousselham, Mathilde Caron, Arsha Nagrani, Cordelia Schmid