AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,805 stories · RSS feed

Hugging Face Trending Papers
Aug 27

Safety by Design: Realized-Cost Constraints for Contextual Bandits with Continuous Actions

The paper introduces a new approach to safety in contextual bandits with continuous actions by enforcing high‑probability constraints on the realized cost rather than on its expectation. It proposes the High‑Probability Constrained UCB algorithm, which balances optimistic reward exploration with pessimistic safety estimation. The authors provide theoretical regret guarantees for linear models and extend the analysis to general function classes, demonstrating experimentally that realized‑cost constraints significantly reduce safety violations compared to expected‑cost baselines.

Hugging Face Trending Papers
Aug 27

DPA-I2P: Depth-Guided Projective Alignment for Image-to-Point-Cloud Registration in Autonomous Driving

The paper introduces DPA-I2P, a depth‑guided projective alignment method for image‑to‑point‑cloud registration in autonomous driving. It employs Ray‑Conditioned Metric Depth Encoding and Projection‑Consistent Vision Lifting to align depth and visual cues geometrically, and uses Cross‑Modal Query Pruning to filter unreliable matches during refinement. Experiments on KITTI and nuScenes show significant improvements, reducing rotation and translation errors by up to 55.6% compared to existing implicit baselines.

arXiv Computer Vision
Aug 27

PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction

PIVOT is a new multi‑trajectory dataset and evaluation framework that captures real‑world scenes with diverse camera paths, preserving both sensor‑derived measured poses and COLMAP‑optimized poses along with calibrated and optimized intrinsics. It defines three benchmark families—seen vs. unseen trajectory generalization, measured vs. optimized pose sensitivity, and calibrated vs. optimized intrinsics sensitivity—and introduces a directed pose‑space Chamfer distance to assess pose coverage. The first version of PIVOT includes five scenes recorded with a DJI Mini 4 Pro and offers an open processing and Nerfstudio‑based evaluation toolchain, revealing a consistent quality gap between held‑out and unseen trajectories and significant sensitivity to pose source and camera intrinsics.

By Mary Raymond
arXiv Computation and Language
Aug 27

Learning What to Share and What to Personalize: Hierarchical Strategy Co-Evolution for Agent Memory

The paper introduces HiPS, a hierarchical strategy co‑evolution framework for memory‑augmented agents that separates memory management into a globally shared foundation and a user‑specific adaptive tier. HiPS uses a Universal Strategy to capture shared principles from cross‑persona trajectories, Persona Delta Distillation to create tailored rules for users deviating from general patterns, and Cross‑Level Rule Flow to dynamically adjust the boundary between global and personal rules. Experiments show that this approach consistently outperforms existing memory‑augmented baselines.

By Yupeng Han, Shuochen Liu, Kai Zhang, Ze Liu, Zhihong Pan, Xianquan Wang
arXiv Computation and Language
Aug 27

AWM: Answerable Working Memory for Long-Document VQA Agents

The paper introduces AWM, a framework that treats the terminal working memory of long‑document VQA agents as an answerable evidence artifact. It proposes a memory‑only answerability diagnostic and incorporates this signal into the GRPO reward, giving higher advantage to trajectories whose final memory can answer the question alone. Experiments on MMLongBench‑Doc and LongDocURL show that AWM‑GRPO boosts final‑answer accuracy by up to 11.9 points and reduces the rate of correct answers that cannot be supported by memory alone.

By Dongzhuoran Zhou, Yuqicheng Zhu, Yule Liu, Zhen Yang, Rui Lu, Yuxiao Dong, Jie Tang, Evgeny Kharlamov
arXiv Machine Learning
Aug 27

TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development

arXiv:2608. 26086v1 Announce Type: new Abstract: Large language models write correct code for isolated problems but remain far weaker at autonomous machine-learning development, where an agent must revise data pipelines, models, and validation over hours of feedback, and on most competitions still finishes below strong human competitors.

By Jiarui Yan, Weiwei Sun, Sijie Li, Wenhan Li, Yiming Yang
arXiv Machine Learning
Aug 27

Cooperative Multi-Agent Reinforcement Learning for Adaptive Aggregation in Semi-Supervised Federated Learning with non-IID Data

The paper introduces pFedMARL, a federated learning framework that uses multi‑agent reinforcement learning with TD3 to dynamically adjust client contributions and personalize models. It applies a server‑side agent to optimize global aggregation and client‑side agents to balance global and local updates, eliminating the need for pre‑training. Experiments on a semi‑supervised audio spectrogram transformer show that pFedMARL outperforms or matches FedAvg, Ditto, and local training across various non‑IID settings and against adversarial clients, improving accuracy, robustness, and fairness.

By Rene Glitza, Luca Becker, Rainer Martin
arXiv Machine Learning
Aug 27

TailSFT: Filtered Fine-Tuning Improves Post-Training Performance

TailSFT is a simple modification to supervised fine‑tuning that filters out already well‑modeled sequences, concentrating learning on the tail of the data distribution. On the OLMo‑3 7B model, this approach improves pass@16 performance on math and coding tasks by up to 17% absolute and yields up to 4% absolute gains in subsequent GRPO reinforcement‑learning runs, with only minimal computational overhead. The authors also provide a lightweight diagnostic to identify settings where TailSFT is most beneficial and argue for a stage‑aware development strategy that evaluates intermediate checkpoints by their support for later training.

By Sadhika Malladi, Samy Jelassi, Dylan Foster, Jordan T. Ash, Akshay Krishnamurthy
arXiv Machine Learning
Aug 27

Generative Modeling: A Review

The paper reviews generative modeling by categorizing generators into three types: those estimating counterfactual outcome distributions in causal inference, those recovering posteriors from simulated parameter–outcome pairs, and those forming predictive outcome distributions. It introduces generative Bayesian computation, a quantile neural network trained on simulated pairs using the pinball loss, which directly targets posterior distributions without requiring invertible architectures or density evaluation. The method is demonstrated on an agent-based Ebola transmission model, showing accurate posterior recovery at lower computational cost than rejection-based simulation inference.

By Maria Nareklishvili, Nick Polson, Vadim Sokolov
arXiv Machine Learning
Aug 27

Scorpio: Serving Right Requests at the Right Time for Heterogeneous SLOs in LLM Inference

Scorpio is an LLM serving system that optimizes for heterogeneous Service Level Objectives (SLOs) such as Time to First Token (TTFT) and Time Per Output Token (TPOT). It uses adaptive scheduling across admission control, queue management, and batch selection, featuring a TTFT Guard that reorders requests by least-deadline-first and rejects unattainable ones, and a TPOT Guard that employs VBS-based admission control and a credit-based batching mechanism. Predictive modules support both guards, and evaluations show Scorpio can increase system goodput by up to 14.4× and improve SLO adherence by up to 46.5% under high load compared to state-of-the-art baselines.

By Yinghao Tang, Tingfeng Lan, Bo Pan, Xiuqi Huang, Hui Lu, Wei Chen
arXiv Machine Learning
Aug 27

Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory

The paper introduces KOPE, an experience‑driven framework that records hardware kernel optimization trajectories in an Experience Graph Memory and uses Active Context Management and Injection to retrieve relevant past decisions under a fixed token budget. KOPE preserves decision order, outcomes, and alternative branches, enabling evidence from completed runs to inform future optimization steps. In experiments, KOPE achieves a 1.54× speedup over the strongest baseline, raises pass rates from 60.0% to 84.6%, and reduces token consumption dramatically, demonstrating the benefits of continual learning from external experience while keeping the foundation model unchanged.

By Siyuan Chen, Runlin Hou, Shenxiu Wu, Yansong Sun, Junming Cao, Yiyu Zhang, Shudi Shao, Junhao Qiu, Zhichao Lu, Qingfu Zhang
arXiv Machine Learning
Aug 27

Beyond Pairwise Feedback: Listwise Vision-Language Supervision for Preference-Based Reward Learning

The paper introduces a novel framework that combines vision‑language model (VLM) generated preferences with the Plackett‑Luce (PL) ranking model for reward learning in reinforcement learning. Unlike traditional pairwise Bradley‑Terry approaches, the PL formulation allows listwise rankings of multiple candidates, enabling the use of different ranking sizes (K = 3, 4, 5). Experiments on Meta‑World manipulation tasks show that PL‑based reward models train robotic policies as effectively as, or better than, pairwise, K‑wise, and RL‑VLM‑F baselines, achieving up to an 86% mean final success rate and matching the Oracle baseline on the Drawer Open task.

By Srivalli Katkuri, Maxwell Kawada, Juan Wachs
arXiv Computer Vision
Aug 27

Lost in Motion: Vision Language Models Fail the Dynamic Gauges Test

The paper titled "Lost in Motion: Vision Language Models Fail the Dynamic Gauges Test" examines how current Vision‑Language Models (VLMs) perform on real‑time gauge reading tasks in industrial settings. Using a new dataset of video sequences featuring circular, linear, and Vernier gauges under varied motion, the study tests state‑of‑the‑art models from GPT‑5 and Gemini 3 families. Results show that these VLMs struggle to accurately interpret needle trajectories and scale semantics, falling short of the traceability and reliability required for safety‑critical monitoring and thus not meeting IEEE and ISO standards for trustworthy synthetic instruments.

By Tairan Fu, Francisco Javier Santos-Mart\'in, Javier Conde, Elena Merino-G\'omez, Pedro Reviriego
arXiv Computer Vision
Aug 27

Fusing Perceptual Vision Experts with Multimodal Large Language Models for Explainable Plant Disease Diagnosis: From Benchmark Imagery to Real-World Robotic Field Validation

The paper introduces the Hybrid Hierarchical Multi-Agent Framework (H$^{2}$MAF), which fuses decision-level outputs from EfficientNet-B3 and ConvNeXt-Tiny with semantic arbitration by multimodal large language models Gemma 4 E4B and Qwen3.5 4B to produce explainable plant disease diagnoses. Evaluated on 14,364 images from PlantDoc and two Cornell robotic field datasets, the framework achieves up to 99.3% accuracy, with Gemma improving PlantDoc accuracy from 63.9% to 68.5% and demonstrating low critical‑risk error. The results highlight the potential of MLLM arbitration for reliable, explainable agricultural AI under real‑world field conditions.

By Ranjan Sapkota, Konstantinos I. Roumeliotis, Pengyao Xie, Nikolaos D. Tselikas, Lirong Xiang, Manoj Karkee
arXiv Computer Vision
Aug 27

Boosting Reasoning in Large Multimodal Models via Activation Replay

The paper introduces Activation Replay, a training‑free method that improves reasoning in post‑trained large multimodal models (LMMs) by replaying low‑entropy activations from the base model’s input context. It shows that Reinforcement Learning with Verifiable Rewards (RLVR) shifts low‑entropy activations and that modulating these activations enhances reasoning across tasks such as mathematics, visual agents, and video reasoning. Experiments demonstrate that Activation Replay outperforms alternatives like high‑entropy replay or direct cross‑model intervention, boosting Pass@K and broadening RLVR’s reasoning coverage.

By Yun Xing, Xiaobin Hu, Qingdong He, Jiangning Zhang, Shuicheng Yan, Shijian Lu, Yu-Gang Jiang
arXiv Computer Vision
Aug 27

VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval

VisDocAgentBench is a closed‑corpus benchmark that evaluates static versus agentic retrieval for visually rich documents, using 2,375 pages from 100 documents and 120 queries that span direct, one‑bridge, and two‑bridge evidence structures. The benchmark includes semantic, relational, and visual queries, full‑document review, and hard‑negative validation. Results show that a strong visual retriever performs well on direct items but poorly on two‑bridge items, while agents improve performance, especially when using visual retrieval and iterative search capabilities.

By Lexiang Hu, Yanzhao Zhang, Mingxin Li, Dingkun Long, Yikang Li, Fuwei Zhang, Yisen Wang, Zhouchen Lin
arXiv Computer Vision
Aug 27

TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding

TAU-Agent is a retrieval‑augmented framework designed for traffic anomaly understanding in transportation videos. It uses a central retrieval agent to coordinate a Video Captioning Tool and an Open‑Vocabulary Tracking Tool, gathering captions, temporal intervals, and object trajectories relevant to a query. The collected evidence, along with sampled frames and the query, is fed into a fine‑tuned vision‑language model that reasons and generates an answer. TAU-Agent was evaluated on the AI City Challenge 2026 benchmarks, achieving notable scores across multiple tracks and ranking second, twelfth, and fifth respectively.

By Yuqiang Lin, Yan Shi, Sam Lockyer, Harish Tayyar Madabushi, Adrian Evans, Wenbin Li, Yinhai Wang, Nic Zhang
arXiv Computer Vision
Aug 27

MAMA-FLUX.2: Image-to-Image Synthesis of Post-Contrast Breast DCE-MRI for the MAMA-SYNTH Challenge

The paper introduces MAMA-FLUX.2, a conditional latent flow‑matching model built on FLUX.2-Klein-4B, designed to synthesize post‑contrast breast DCE‑MRI from pre‑contrast images for the MAMA‑SYNTH challenge. It encodes the pre‑contrast image as spatial conditioning and predicts the flow field for the post‑contrast latent, adapting the pretrained model with LoRA fine‑tuning and a regional training objective that combines global flow matching, tumor‑region supervision, and stable foreground regularization. Ablation studies show that moderate tumor and stable‑foreground weighting improves the balance between image fidelity and tumor‑region accuracy, with the best configuration achieving a strong trade‑off using LoRA rank 64/64, MHA_max 25, λ_tumor 0.25, and λ_stable 0.1.

By Kamil Kwarciak, Marek Wodzinski
arXiv Computer Vision
Aug 27

AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research

AdaVDR is an adaptive video deep research agent that selects and reflects on tool usage based on the task and the model’s capabilities. It constructs a specialized data pipeline to generate high‑quality QA pairs and uses model‑conditioned filtering to remove unnecessary tool calls. The agent is trained with supervised fine‑tuning and reinforcement learning, achieving top performance on the VDR‑EE benchmark and significant gains on VideoDR.

By Xintong Zhang, Xiaomeng Fan, Shilin Yan, Ekko He, Zicheng Liu, Zijian Zou, Guannan Zhang, Yuwei Wu, Zhi Gao, Hongwei Xue
arXiv Computation and Language
Aug 27

Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG

Corpus2Skill is a retrieval architecture that transforms an enterprise knowledge base into a hierarchical skill directory, enabling an LLM agent to navigate from high-level summaries to specific documents and backtrack when necessary. On an enterprise customer‑support benchmark, it outperforms single‑shot dense, hybrid, hierarchical‑retrieval, and agentic RAG baselines in answer quality and grounding, with a moderate cost tradeoff. An eleven‑dataset study shows that corpus navigation excels on single‑domain corpora with a recoverable topical taxonomy but is less effective on open‑domain factoid pools or homogeneous‑tabular corpora, providing a design guideline for knowledge‑grounded systems.

By Yiqun Sun, Pengfei Wei, Lawrence B. Hsieh