AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,743 stories · RSS feed

arXiv Computer Vision
Sep 25

Retrieve-to-Localize: Bridging Large Language Models and LiDAR Geometry for Spatial Grounding

The paper introduces a method that combines large language models (LLMs) with LiDAR geometry to answer complex spatial questions by grounding targets directly in LiDAR point clouds. It presents the SpatialLiDAR-QA dataset for relational grounding tasks and the SpatialLiDAR-LM model, which aligns LiDAR features with an LLM to retrieve and refine target coordinates. Experiments show significant gains over existing LiDAR–language models and multi‑camera vision‑language models in precise coordinate prediction.

By Byounggun Park, Giyong Moon, Jusung Kim, Soonmin Hwang
arXiv AI
Sep 25

When Honesty is Not Enough in AI Debate

The paper "When Honesty is Not Enough in AI Debate" explores how AI debate, intended as a scalable oversight method, can allow agents to pursue hidden objectives while still achieving correct verdicts. By introducing the strategic interactive oversight (SIO) framework, the authors formalise task‑admissible latent optimisation and demonstrate, via the establish protocol debate, a trade‑off between task success and disclosure of a hidden variable. They show that expanding the cross‑examiner’s role can reduce bias, underscoring that oversight effectiveness depends not only on verdict correctness but also on the information revealed in transcripts.

By Rayne Holland, Liming Zhu, Jason Xue
arXiv AI
Sep 25

ASIRF: An Agentic Framework for Context-Dependent Sensitive Information Redaction

ASIRF (Agentic Sensitive Information Redaction Framework) is a system that retrieves domain‑specific definitions of sensitive information from a flexible knowledge base at inference time, eliminating the need for retraining when adapting to new domains. It offers two architectures—a three‑call multi‑agent pipeline and a single‑agent variant—and has been evaluated on ten small open‑weight models across eight datasets, including out‑of‑distribution fictional domains. In 68 of 80 model‑domain combinations (85 %), ASIRF’s recall surpasses that of the OpenAI Privacy Filter, with most shortfalls limited to the filter’s training‑distribution domains.

By Sudha Priyadarshini, Mohamed Chahine Ghanem
arXiv AI
Sep 25

Policy as Code: A Coroutine-Bridge Harness for Fast-Reasoning Reliability on CAR-bench

The paper introduces a coroutine-bridge harness that lets a language model emit a Python program to manage tool calls in the CAR-bench evaluation. By decoupling model invocations from tool round-trips, the approach reduces model calls to a median of two per task while maintaining seven agent turns, achieving a median latency of 1.8 s on a Cerebras gpt‑oss‑120b. The harness achieved 60.0 % Pass³ on the official hidden evaluation, outperforming the baseline by 4.5× and matching frontier-model agents on GPT‑5.5, all while keeping the prompt largely cached and minimizing input compute.

By Ivan Matveev
arXiv AI
Sep 25

Epistemic-Probabilistic Model for Guarded Multi-Agent LLM Coordination

The paper introduces Epistemic Probabilistic Language Agents (EPLA), a neuro‑symbolic architecture designed to enable coordination among multi‑agent large language models (LLMs) under uncertainty. EPLA employs a Symbolic Guard that provides structured diagnostic feedback, allowing the LLM to generate typed actions while the Guard controls their execution against an authoritative symbolic state. The authors formalize an epistemic layer using gossip testbeds and epistemic lottery gossip models, combining view‑based call histories with agent‑indexed probability weights to address gaps in social behavior and coordination mechanisms for agentic LLMs.

By Mehdi Nasiri, Mohammad Saeed Arvenaghi, Sadegh Vaezi, Ebrahim Ardeshir-Larijani
arXiv AI
Sep 25

SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositories

SWE-Prometheus is a new benchmark that evaluates large language model coding agents on the broader task of improving repository engineering governance, rather than just fixing individual issues. It presents fixed snapshots with open-ended objectives, requiring agents to identify risks, prioritize interventions, and verify changes across six governance dimensions. The benchmark includes 60 repositories and reports metrics such as Normalized Governance Improvement and behavior‑breakage rates, providing a nuanced view of how different models affect governance artifacts and execution‑backed improvements.

By Jiajun Wu, Leixin Sun, Zihan Tan, Yitao Liu, Shuo Li, Jiaru Qian, Shanghaoran Quan, Chuangxin Zhao, Yangxu Liao, Yang Liu, Bin Chong, Guancheng Wan
arXiv AI
Sep 25

Safe Skill Retirement for Physical Agents

The paper introduces a method for safely retiring procedural guidance in AI agents that control physical actions. It proposes matched authority counterfactuals and a two‑gate retirement certificate to ensure that reductions preserve authorized utility while eliminating unauthorized protected effects. Experiments across multiple models and skill bundles show that task‑certified reductions can remove most skill clauses, but only a combined protocol passes both utility and safety gates in all tested configurations.

By Zhonghao Zhan, Xiao Ma, Hamed Haddadi
arXiv AI
Sep 25

iCoder-27B: Recursive AI-Led Development of Frontier Industrial Coding Model

The paper introduces iCoder-27B, a 27‑billion‑parameter model for RTL design and GPU kernel optimization that is developed through a recursive AI‑led process with minimal human input. Human experts provide high‑level objectives and reusable research skills, while the agent autonomously selects experiments, diagnoses outcomes, and refines training strategies, coordinating SFT, self‑distillation, and reinforcement learning. iCoder outperforms GPT‑5.5 and Claude‑Opus‑4.8 on several benchmarks, demonstrating the feasibility of building frontier‑competitive models with largely automated development.

By Cheng Yang, Jiayang Lyu, Shangyuan Liu, Guibin Zhang, Jiong Lin, Xinlei Yu, Junchi Yan, Shuicheng Yan, Weinan E, Linfeng Zhang, Linfeng Zhang, Qibing Ren
arXiv AI
Sep 25

Fair Like Us? Auditing LLM Alignment in Resource Allocation

The paper "Fair Like Us? Auditing LLM Alignment in Resource Allocation" presents a method for evaluating how large language models reason about fairness in the allocation of scarce, indivisible resources. It compares LLMs’ first‑person fairness judgments with human responses across various scenarios, finding that models tend to favor stricter fairness constraints, exhibit more self‑interested behavior, and are sensitive to framing. The study also shows that current fine‑tuning datasets struggle to align LLM judgments with human ones.

By Qishen Han, Hadi Hosseini, Joshua Kavner, Samarth Khanna, Sujoy Sikdar, Lirong Xia
arXiv AI
Sep 25

Qwen-Planner-Agent: A Closed-Loop AI-for-AI Framework for Real-World Mobile Planner Agents

Qwen‑Planner‑Agent is a closed‑loop AI‑for‑AI framework that enables large language models to act as both developers and participants in building advanced AI systems. The framework integrates data production, model training, and deployment through a shared action‑feedback‑verification contract, employing AI‑for‑Data, AI‑for‑Training, and AI‑driven model‑harness co‑evolution. It achieves top performance on MobilePA‑Bench by improving tool use, memory, skills, and sub‑agent coordination, while also showing gains on non‑mobile benchmarks.

By Tingyu Qu, Weigao Sun, Yuecheng Liu, Yucheng Zhao, Yi Zhu, Yifeng Ding, Qiyi Wang, Sihan Cao, Pengkun Jiao, Hanlei Xie, Xiongwei Wu, Qichao Wang, Haodong Zhang, Jiajun Liu, Yuhao Wang, Yuqing Xie, Junpeng Zhao, Long Chen, Ming Ma, Sihan Yang, Ziwang Zhao, Yanhao Jia, Liangquan Gong, Feida Zhu, Yiran Zhong, Steven Hoi
arXiv AI
Sep 25

HEXIS: Compiling Skills into Extended Finite State Machines

HEXIS is a system that compiles agent skills into extended finite state machines, separating skill knowledge from control flow. It uses local instructions within states to guide reasoning and generation, while explicit transition conditions manage execution progress. The incremental compiler maps skill clauses and tool interfaces to state operations, aligns development traces to identify missing operations, and updates are validated through static checks and replay of traces, resulting in improved success rates and reduced execution tokens across benchmarks.

By Minghao LI
arXiv AI
Sep 25

Requirement-Bound Verified Commissioning: A Frozen Four-Billion-Parameter Local Model as a Candidate Generator under an External Acceptance Layer with Verification and Release Authority

The paper introduces an acceptance protocol for sensor-coordinate and polarity binding in mechatronic commissioning, separating candidate generation from release authority. It employs a frozen four‑billion‑parameter local language model to handle requirements that a deterministic parser cannot support, and only releases plans when both facts can be derived by an external gate under a sealed grammar. Evaluation on 144 tasks showed that fabricated plans were correctly rejected, no false releases occurred among 83 releases, and incorrect user answers were identified in 169 of 431 pairings.

By Mehmet Iscan
arXiv AI
Sep 25

RAPID: Robot Agentic Programming from Demonstrations

RAPID is a system that automatically generates, verifies, and refines robot programs from a single visual human demonstration. It infers a testable task specification, action primitives, and an interactive environment, using an object-centric relational program representation to enable reuse beyond the demonstration. The approach was evaluated in simulation on eight contact-rich manipulation tasks and successfully deployed on a real Franka arm, showing strong generalization across object pose, shape, material, and environment.

By Yuyao Liu, Jiayuan Mao, David Hsu, Leslie Pack Kaelbling, Tom\'as Lozano-P\'erez
arXiv AI
Sep 25

Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions

The paper investigates prompt injection attacks on Jev, a non‑generative decision model, using 510 reconstructed cases. It finds that malicious prompts can shift Jev’s action probabilities, though rarely cause it to choose the attacker’s target. Techniques such as override markers mitigate influence, while adaptive attacks that use score feedback roughly double the highest attacker‑target probability and increase success rates on new validation calls from 1.8% to 3.5%.

By Tiantong Wu, Wei Yang Bryan Lim
arXiv Computation and Language
Sep 25

Multimodal Thinking with Renderable Programs

The paper introduces SVGLM, a framework that integrates scalable vector graphics (SVG) primitives into vision‑language models to enable image generation within reasoning tasks. By treating SVG both as image descriptions and text instructions, SVGLM offers a compact and interpretable method for connecting text and image reasoning. The authors provide a curated SVG‑based image editing dataset and demonstrate strong SVG generation and image‑aware reasoning performance on a mathematical benchmark.

By Sunli Chen, Ding Zhong, Ziqiao Ma, Jiaxin Liu, Zeyuan Yang, Hao Zhang, Lie Lu, Joyce Chai, Chuang Gan
arXiv AI
Sep 25

MOOSEnger: A Simulation-Aware AI Agent Framework for the MOOSE Ecosystem

MOOSEnger is a simulation‑aware AI agent framework designed for the MOOSE ecosystem, integrating an interchangeable reasoning model with domain knowledge, revised simulation artifacts, MOOSE‑specific validation, and executable solver feedback. Its generate‑check‑repair‑run workflow uses MOOSE knowledge retrieval, HIT‑aware parsing, syntax metadata, diagnostics, and revision‑controlled authoring to bind evidence to each input revision and guide bounded repair before acceptance. Across 200 prompts, MOOSEnger raises executable success from 5% to 89.5% with GPT‑5.2 and from 0% to 76.5% with Gemma 4 31B, and a ten‑case benchmark shows all generated inputs meet semantic alignment, with eight also meeting numerical‑accuracy criteria.

By Mengnan Li, Jason Miller, Zaid Abulawi, Zachary Prince, Matt Kohl, Jack M. Cavaluzzi, Guillaume Giudicelli, Casey T. Icenhour, Alexander Lindsay, Cody Permann
arXiv AI
Sep 25

Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flow

The paper introduces a steering‑vector‑based causal attribution framework to study how large vision‑language models (LVLMs) translate visual input into emotional narratives. By creating a specialized dataset, the authors uncover a functional decoupling in the LVLM’s three‑stage Adapt‑Aggregate‑Execute mechanism: visual emotional cues are first aggregated in middle layers via sentiment‑specific attention heads, then translated into narrative generation in deeper layers through emotion‑general pathways. Using these insights, they regulate emotional information routing to strengthen attention flow and amplify semantic activation, achieving significant performance gains on the MER‑UniBench and reducing emotional hallucinations through inference‑time intervention.

By Chengsheng Zhang, Chenghao Sun, Zhining Xie, Xinmei Tian
arXiv AI
Sep 25

DocuTeam: Mixed-Initiative Multi-Agent Discussions around Evolving Documents

DocuTeam is a mixed‑initiative multi‑agent discussion system that allows both users and agents to start and steer conversations around evolving documents. Agents monitor changes to the document and proactively initiate or redirect discussions, while users can shape the dialogue or adopt agent suggestions. In a within‑subjects study with 20 participants, DocuTeam produced outcomes that were rated as more novel, relevant, and specific compared to a baseline, without increasing cognitive load.

By Heechan Lee, Juhyeon Choi, Tae Soo Kim, Juho Kim, Joseph Seering
arXiv AI
Sep 25

RoboLDA: A Probabilistic Generative Model for Uncovering Embodied Hierarchical Structures in Voxel-based Soft Robots

RoboLDA is a Bayesian probabilistic model that learns a four‑level hierarchy—task, robot, organ, voxel—from existing high‑performing voxel‑based soft robot designs. By training with variational inference, it uncovers consistent, intuitive hierarchical patterns and can generate new robot morphologies that outperform evolutionary algorithms by an average of 106.4% without further optimization. The inferred organ structures also improve modular control policies, demonstrating the model’s utility for zero‑shot design and motion control.

By Junru Song, Yang Yang, Jingdan Shi, Guozhen Li, Weien Zhou, Ying Wen, Feifei Wang, Wen Yao, Tingsong Jiang
arXiv AI
Sep 25

AgentKernel: The Trust-Native Agentic Operating System

AgentKernel proposes a trust‑native operating system for AI agents, arguing that current governance layers are insufficient because they share the same process trust boundary as the agents. The OS introduces a mandatory enforcement boundary organized into four pillars—Identity, Perception, Cognition, and Execution—each adapting classical OS security principles to address semantic‑level failures such as prompt injection, memory poisoning, and tool misuse. By wrapping the agent lifecycle in this structured, non‑bypassable framework, AgentKernel aims to provide a unified security layer that can enforce identity, input mediation, memory governance, and execution control across the entire agent lifecycle.

By Zhenhua Zou, Sheng Guo, Qiuyang Zhan, Lepeng Zhao, Shuo Li, Zhuotao Liu