AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,805 stories · RSS feed

arXiv Machine Learning
Aug 27

Beyond Pairwise Feedback: Listwise Vision-Language Supervision for Preference-Based Reward Learning

The paper introduces a novel framework that combines vision‑language model (VLM) generated preferences with the Plackett‑Luce (PL) ranking model for reward learning in reinforcement learning. Unlike traditional pairwise Bradley‑Terry approaches, the PL formulation allows listwise rankings of multiple candidates, enabling the use of different ranking sizes (K = 3, 4, 5). Experiments on Meta‑World manipulation tasks show that PL‑based reward models train robotic policies as effectively as, or better than, pairwise, K‑wise, and RL‑VLM‑F baselines, achieving up to an 86% mean final success rate and matching the Oracle baseline on the Drawer Open task.

By Srivalli Katkuri, Maxwell Kawada, Juan Wachs
arXiv Computer Vision
Aug 27

Training-Free Agentic Computer Vision for Structural Component Detection in 2D Structural Framing Plans

The paper presents a training‑free, agentic computer‑vision system that converts 2D structural framing plan PDFs into editable finite‑element model drafts. It uses a deterministic stage to extract geometric primitives, estimate scale, and recognize five entity classes via a drafting grammar, followed by an agentic stage that applies typed corrections and fail‑closed transactions. Evaluation on a 100‑plan benchmark shows high precision and recall across columns, beams, walls, braces, and openings, with scale estimates within 0.1% of reference.

By Mohammad Talebi-Kalaleh, Qipei Mei
arXiv Computation and Language
Aug 27

Trace Integrity for LLM Data Agents: A Vision for Auditable Structured Reasoning in Real-World Systems

The paper argues that answer accuracy alone is insufficient for evaluating large language model (LLM) data agents, especially in structured-data tasks where a correct answer can be produced by an invalid trace. It introduces Trace Integrity as a reliability criterion that ensures the computation behind an answer is explicit, executable, schema-valid, operator-faithful, replayable, answer-consistent, and auditable. The authors operationalize this concept with execution contracts and present the CAIT (Correct Answer / Invalid Trace) Rate to quantify how often answer-only evaluations mistakenly reward unsupported outputs, demonstrating that accuracy, trace validity, and silent-failure risk are distinct signals.

By Srimonti Dutta, Akshata Kishore Moharir
arXiv Statistics ML
Aug 27

Schr\"odinger Bridges over Kinetic Swarming Models

The paper studies finite‑horizon minimum‑energy steering of inertial swarms under stochastic disturbances, focusing on mean‑field models with Cucker–Smale alignment or Morse attraction–repulsion interactions. It formulates the problem as a Schr"odinger bridge, deriving nonlinear, time‑symmetric optimality systems and proposing nested fixed‑point schemes for numerical solution. Numerical experiments demonstrate that the optimal corrective drift can either exploit or counteract the natural interaction forces, depending on their alignment with the steering objective.

By Asmaa Eldesoukey, Md Zulfiqur Haider, Italo Napolitano, Yongxin Chen, Abhishek Halder
arXiv Machine Learning
Aug 27

Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory

The paper introduces KOPE, an experience‑driven framework that records hardware kernel optimization trajectories in an Experience Graph Memory and uses Active Context Management and Injection to retrieve relevant past decisions under a fixed token budget. KOPE preserves decision order, outcomes, and alternative branches, enabling evidence from completed runs to inform future optimization steps. In experiments, KOPE achieves a 1.54× speedup over the strongest baseline, raises pass rates from 60.0% to 84.6%, and reduces token consumption dramatically, demonstrating the benefits of continual learning from external experience while keeping the foundation model unchanged.

By Siyuan Chen, Runlin Hou, Shenxiu Wu, Yansong Sun, Junming Cao, Yiyu Zhang, Shudi Shao, Junhao Qiu, Zhichao Lu, Qingfu Zhang
arXiv Computation and Language
Aug 27

Belief Cascades Drive Persuasion in LLM Agent Networks

The paper introduces a controlled testbed to study how goal‑directed persuaders shift stances in networks of large language model agents, using real‑world ego‑network topologies. Experiments across four LLM backbones, five graph structures, and 55 policy statements show that persuasion dynamics depend on topology, competition, topic, and model prior. The study finds that direct exposure predicts stance change, peer relays have measurable influence, and that post‑text analysis alone misses important movement, highlighting the need to evaluate multi‑agent persuasion through trajectory‑level processes, belief probes, exposure provenance, and action logs.

By Haoyi Qiu, Genglin Liu, Pranav Narayanan Venkit, Kung-Hsiang Huang, Saadia Gabriel, Chien-Sheng Wu, Nanyun Peng
arXiv Computation and Language
Aug 27

TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving

TOPAS is a Task‑Oriented Prefix‑Aware Scheduler designed for multi‑agent large language model serving. It jointly decides which agent prefixes to retain in a shared key‑value cache and which requests to schedule, balancing the reduction of each task’s longest remaining service path against the benefit of downstream prefix reuse while accounting for movement and preemption costs. Experiments on synthetic DAGs and MetaGPT software‑development workflows show that TOPAS can reduce mean and p99 job completion times by up to 39.8%/49.4% and 22.0%/26.6% respectively compared to the best baselines.

By Hongqiu Ni, Han Tian, Chi Zhang, Guopeng Li, Haisheng Tan
arXiv Computation and Language
Aug 27

PlanSightRAG: A Visual-First Multimodal RAG for Automating Question Answering and Compliance Checking for Civil Standard Plans

PlanSightRAG is a visual-first multimodal retrieval‑augmented generation system designed to automate question answering and compliance checking of civil standard plans. It processes plan imagery directly, using a ColNomic‑3B multi‑vector retrieval engine, an agentic Planner‑Retriever‑Auditor‑Synthesizer, and MaxSim heatmaps to provide an evidence trail. The system achieves high recall on a new 4,056‑pair benchmark from five state DOTs, and demonstrates near‑perfect verdict accuracy on synthetic compliance drawings when a rule threshold is supplied, outperforming OCR‑based baselines.

By Nabaraj Subedi, Shuvo Dip Datta, Ahmed Abdelaty, Shivanand Venkanna Sheshappanavar
arXiv Computation and Language
Aug 27

FrontierChallenge: Evaluating Scientific Workflow Completion

FrontierChallenge is a cross‑domain benchmark that releases 300 end‑to‑end scientific workflows, of which 97 are evaluated in this study. The benchmark covers diverse fields such as quantum chemistry, molecular dynamics, materials characterization, analytical chemistry, life science, and electrochemistry/environment, and requires agents to produce a bundle of fixed scientific deliverables. Twelve frontier models were tested, and the best configurations completed only 20 of the 97 tasks, achieving a 20.6% pass rate; high partial scores and confident completion claims often did not translate into full delivery, especially in analytical chemistry and electrochemistry/environment.

By Liangcai Su, Zhaopeng Feng, Zhuo Chen, Zhen Zhang, Xiang Lin, Ruilin Li, Handuo Zhang, Ning Wang, Kailong Wen, Yueqi Guo, Feng Xing, Yiling Guo, Chenxiong Qian, Simon Shaolei Du, Lidong Bing, Xinyu Wang
arXiv Machine Learning
Aug 27

TailSFT: Filtered Fine-Tuning Improves Post-Training Performance

TailSFT is a simple modification to supervised fine‑tuning that filters out already well‑modeled sequences, concentrating learning on the tail of the data distribution. On the OLMo‑3 7B model, this approach improves pass@16 performance on math and coding tasks by up to 17% absolute and yields up to 4% absolute gains in subsequent GRPO reinforcement‑learning runs, with only minimal computational overhead. The authors also provide a lightweight diagnostic to identify settings where TailSFT is most beneficial and argue for a stage‑aware development strategy that evaluates intermediate checkpoints by their support for later training.

By Sadhika Malladi, Samy Jelassi, Dylan Foster, Jordan T. Ash, Akshay Krishnamurthy
arXiv Machine Learning
Aug 27

Scorpio: Serving Right Requests at the Right Time for Heterogeneous SLOs in LLM Inference

Scorpio is an LLM serving system that optimizes for heterogeneous Service Level Objectives (SLOs) such as Time to First Token (TTFT) and Time Per Output Token (TPOT). It uses adaptive scheduling across admission control, queue management, and batch selection, featuring a TTFT Guard that reorders requests by least-deadline-first and rejects unattainable ones, and a TPOT Guard that employs VBS-based admission control and a credit-based batching mechanism. Predictive modules support both guards, and evaluations show Scorpio can increase system goodput by up to 14.4× and improve SLO adherence by up to 46.5% under high load compared to state-of-the-art baselines.

By Yinghao Tang, Tingfeng Lan, Bo Pan, Xiuqi Huang, Hui Lu, Wei Chen
arXiv Machine Learning
Aug 27

Generative Modeling: A Review

The paper reviews generative modeling by categorizing generators into three types: those estimating counterfactual outcome distributions in causal inference, those recovering posteriors from simulated parameter–outcome pairs, and those forming predictive outcome distributions. It introduces generative Bayesian computation, a quantile neural network trained on simulated pairs using the pinball loss, which directly targets posterior distributions without requiring invertible architectures or density evaluation. The method is demonstrated on an agent-based Ebola transmission model, showing accurate posterior recovery at lower computational cost than rejection-based simulation inference.

By Maria Nareklishvili, Nick Polson, Vadim Sokolov
arXiv Computer Vision
Aug 27

3DGAA: Realistic and Robust 3D Gaussian-based Adversarial Attack for Autonomous Driving

The paper presents 3DGAA, a fabrication-first framework that generates view-consistent, geometry-preserving adversarial wraps for vehicles using 3D Gaussian splatting optimization. It ensures consistency across viewpoints, illumination, and occlusion while limiting changes to vehicle geometry, producing realistic print-only textures that significantly reduce detection confidence and average precision in simulations and physical tests. Ablation and efficiency studies analyze the impact of physical filtering, augmentation, and shape-consistency regularization, and the method demonstrates robustness against common preprocessing defenses and cross-detector transferability.

By Yixun Zhang, Lizhi Wang, Junjun Zhao, Wending Zhao, Feng Zhou, Yonghao Dang, Jianqin Yin
arXiv Computer Vision
Aug 27

Synergistic Modality-and-Slice Memory Framework for Cross-Modal 3D Brain Tumor Segmentation

The paper introduces MSM‑Seg, a dual‑memory segmentation framework for 3D multi‑modal brain tumor segmentation. It combines a modality‑and‑slice memory attention module to capture cross‑modal and spatial‑slice dependencies, a multi‑scale category‑agnostic prompt encoder for whole‑tumor guidance, and a modality‑adaptive fusion decoder to integrate complementary decoding information. Experiments on various MRI datasets show that MSM‑Seg surpasses state‑of‑the‑art methods for metastases and glioma tumor segmentation.

By Yuxiang Luo, Qing Xu, Hai Huang, Yuqi Ouyang, Xiangjian He, Zhen Chen, Wenting Duan, Jiebo Luo
arXiv Computer Vision
Aug 27

Latent Chain-of-Thought World Modeling for End-to-End Driving

Latent-CoT-Drive (LCDrive) is a vision‑language‑action model for autonomous driving that replaces natural‑language chain‑of‑thought reasoning with a latent language capturing possible outcomes of driving actions. The model interleaves action‑proposal tokens, aligned with the model’s output actions, and world‑model tokens grounded in a learned latent world model to reason about future outcomes. After a supervised cold‑start using ground‑truth future rollouts, LCDrive is further refined with closed‑loop reinforcement learning, achieving faster inference, higher‑quality trajectories, and greater gains from interactive RL than both non‑reasoning and text‑reasoning baselines on a large‑scale end‑to‑end driving benchmark.

By Shuhan Tan, Kashyap Chitta, Yuxiao Chen, Ran Tian, Yurong You, Yan Wang, Wenjie Luo, Yulong Cao, Philipp Krahenbuhl, Marco Pavone, Boris Ivanovic
arXiv Machine Learning
Aug 27

Large Language Model Few-Shot Prompting with Dilemma Training Outperforms Human Surrogates in Predicting Patient Preferences

The paper introduces P4-DT, a personalized patient preference predictor that uses dilemma training to elicit context‑dependent decision reasoning. In a study of 12 patient‑surrogate pairs, P4‑DT achieved 81.7% accuracy in predicting patient treatment choices, outperforming unassisted surrogates (55.0%) and surrogates aided by a simpler P4 model (61.7%). The authors show that incorporating contextual scenarios and open‑ended text into prompts improves accuracy by 15 percentage points over static value ratings.

By Natasha Ureyang, Sebastian Porsdam Mann, Yuxin Liu, Zuriel Hassirim, Melanie Almonte, Wenhao Chen, Joyce Ng, Thant Nay Lin, Aung Thiha, Gerald CH Koh, Brian David Earp, Pin Sym Foong
arXiv Computer Vision
Aug 27

TASE: Truncation-Aware Semantic Embeddings for 3D Scene Understanding and Editing

TASE introduces a truncation‑aware embedding space that projects pretrained 2D semantic features into 3D scene representations, allowing flexible and controllable editing. The method optimizes feature channels so that fewer channels yield abstract semantics while more channels preserve detail, and it enforces multi‑view consistency with a scale‑ and translation‑equivariant loss. A finetuning stage for the editing diffusion model further reduces artifacts from geometric changes, achieving competitive performance and outperforming prior methods on large‑scale geometric edits.

By Tim-Felix Faasch, Jochen Kall, Lucas Nunes, Jens Behley, Cyrill Stachniss
arXiv Computation and Language
Aug 27

EduDial: Constructing a Large-scale Multi-turn Teacher-Student Dialogue Corpus

EduDial is a large-scale multi-turn teacher‑student dialogue corpus covering 345 core knowledge points and 34,250 dialogue sessions, designed around Bloom’s taxonomy and ten questioning strategies such as situational, ZPD, and metacognitive questioning. The dataset includes differentiated teaching strategies for students at varying cognitive levels to provide targeted guidance. Using EduDial, the authors trained EduDial‑LLM 32B and introduced an 11‑dimensional evaluation framework that measures teaching quality and content quality, showing that most mainstream LLMs struggle with student‑centered teaching while EduDial‑LLM outperforms all baselines across all metrics.

By Shouang Wei, Min Zhang, Xin Lin, Bo Jiang, Zhongxiang Dai, Kun Kuang
arXiv Machine Learning
Aug 27

Regularized Latent Dynamics Prediction is a Strong Baseline For Behavioral Foundation Models

The paper introduces Regularized Latent Dynamics Prediction (RLDP), a method that adds orthogonality regularization to self‑supervised next‑state prediction in latent space. RLDP maintains feature diversity, matching or surpassing complex representation learning approaches for zero‑shot reinforcement learning. It also performs robustly in low‑coverage data settings where prior methods fail.

By Pranaya Jajoo, Harshit Sikchi, Siddhant Agarwal, Amy Zhang, Scott Niekum, Martha White
arXiv Computation and Language
Aug 27

ClueWeaver: Reward-Guided Dual-Agent Evidence Reasoning for Compact LLMs on Literary Long Narratives

ClueWeaver is a dual-agent framework designed to enable compact, locally deployable language models to answer questions about long literary narratives. The Finder agent retrieves passages that contain answer-critical clues, while the Interpreter agent derives the answer from those passages, generates rationales with paragraph-ID citations, and performs self-calibration for high-risk questions. Both agents are trained with reward-guided reinforcement learning to prioritize evidence retention, correctness, grounding, and concise explanations, resulting in improved performance and inspectability over end-to-end prompting.

By Jihao Zhu, Zhiwei Yang, Wenxiao Zhang, Junqian Zhao, Qi You, Fangqi Wang, Zheyuan Deng, Hanzhe Yang, Yu Liu, Jin B. Hong