AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,750 stories · RSS feed

arXiv AI
Sep 2

ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning

ARISE‑RL is a full‑cycle self‑evolution framework that couples a task/rubric generator with a reasoning solver, enabling open‑ended agents to learn via reinforcement learning without relying on gold answers. The generator creates tool‑grounded rubric criteria and rewards itself for producing valid, intermediate‑difficulty tasks, while the solver improves through fine‑grained rubric satisfaction signals and multi‑step reasoning. The framework also introduces Reward‑Gated Self‑Evolution Distillation to selectively distill memory‑augmented policies, reducing distribution mismatch, and it is evaluated on the new ECR‑Bench rubric suite, achieving state‑of‑the‑art performance across all benchmarks.

By Fanrui Zhang, Ruixue Ding, Qiang Zhang, Xi Chen, Boli Chen, Shihang Wang, Qiuchen Wang, Hongmin Zhan, Jinxin Bian, Li xingchao, Peijin Zheng, Hao cheng, Pengjun Xie, Kaipeng Zhang, Jiawei Liu, Zheng-Jun Zha
arXiv AI
Sep 2

Good Memory Has ECC: Evaluating the Memory of Vision-Language Models Beyond Accuracy

The paper introduces ECCBench, a new benchmark for evaluating the memory of vision‑language models beyond simple accuracy. ECCBench measures memory along three axes—efficiency (FLOPs needed to retrieve information), compression (how well compressible inputs are remembered), and calibration (whether the model appropriately signals uncertainty). The study finds that pretrained VLMs compress text but not video, are poorly calibrated, and that some non‑Transformer backbones outperform RoPE Transformers in compression‑calibration trade‑offs, indicating potential benefits for long‑horizon agents.

By Shmuel Berman, Jia Deng
arXiv AI
Sep 2

Are We There Yet? Assessing Computer-Use Agents for Blind Users' Accessible Interaction with Desktop Applications

The study evaluates computer-use agents (CUAs) for blind users by conducting a three‑week diary study with eight participants using the OLLA prototype. Across 1,258 commands in 12 desktop applications, GPT‑5 achieved the highest success rate of 52.5%, while analysis uncovered failures in grounding, planning, constraint‑tracking, and termination. Interviews highlighted additional needs beyond automation for blind users.

By Satwik Ram Kodandaram, Monalika Padma Reddy, Xiaojun Bi, Jiawei Zhou, I. V. Ramakrishnan, Vikas Ashok
arXiv AI
Sep 2

Agentic programs: an emerging form of scientific software in computational materials science

The article introduces the concept of agentic programs—scientific software that blends deterministic algorithms with bounded large‑language‑model (LLM) judgment, task‑specific verification, episodic maturation, and full delegation in production. It argues that recent LLM‑based agents enable this new form of computational materials science software. The authors illustrate the idea with DeMARS, an agentic program designed to build atomistic models from experimentally measured disordered crystal structures.

By Yunsung Lim, Haekwan Jeon, Jaesun Kim, Jisu Kim, Seungwu Han
arXiv AI
Sep 2

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

EmbodiedSkills is a unified framework that treats each skill decision as an execution proposal, checking prerequisites and verifying outcomes during long‑horizon vision‑language‑action tasks. It connects high‑level skill selection, bounded low‑level VLA execution, and post‑action verification through a fixed executable‑skill interface, enabling easy replacement of low‑level policies and recording of structured trajectories for supervision and adaptation. Instantiated with Qwen3‑VL and OpenPI/pi0.5 on RoboTwin 2.0 and LIBERO, the framework achieves high success rates (86.20% and 97.40% respectively) and demonstrates effective memory‑dependent task performance.

By Wei Wang, Wenqiao Zhang, Yutong Lin, Yuqian Yuan, Tianwei Lin, Jinhao Mao, Zhenxuan Fan, Mingjian Gao, Yang Dai, Wentong Li, Zheqi Lv, Zheng Dong, Yingjie Niu, Jiaqi Zhu, Jun Xiao, Chao Li, Yueting Zhuang
arXiv Machine Learning
Sep 2

Field-Aware Agent Skill Retrieval

Field-Aware Agent Skill Retrieval explores how keeping the distinct fields of skill documents—such as name, description, and body—separate can improve retrieval performance. By computing sparse and dense similarities for each field independently and combining them with either uniform weights or a small MLP, the authors achieve higher Recall@10 scores on two benchmarks, SkillRet and SRA-Bench. The study shows that the advantage of field-aware representation grows as the skill bank expands, indicating its importance for large-scale lifelong learning agents.

By Paimon Goulart, Liang Wu, Kelly Wan, Evangelos E. Papalexakis, Liangjie Hong
arXiv Computation and Language
Sep 2

Creative Generation via Multi-Agent Debate: Does Debate Suppress Diversity?

The paper investigates the use of Multi-Agent Debate (MAD) for creative generation tasks such as narrative writing and scientific ideation. It finds that MAD’s convergence-driven design suppresses output diversity across independent runs, creating a trade-off with creative tasks. To address this, the authors propose Creative-MAD, which introduces Cognitive Lens Assignment and Embedding-based Peer Selection to preserve agent divergence, and demonstrate that it improves lexical and semantic diversity while maintaining quality.

By Tien Anh Nguyen, Khanh-Binh Nguyen, Van Dai Do, Svetha Venkatesh, Hung Le
arXiv Machine Learning
Sep 2

Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC

The paper introduces Solver-Gradient Guided Reinforcement Learning (SG‑RL), a method that augments standard RL with bounded gradients from a differentiable MPC solver to adapt cost‑function weights online. SG‑RL integrates solver‑gradient guidance into PPO through actor‑update scaling, policy loss, advantage estimation, and value‑function learning, achieving comparable or superior closed‑loop performance while requiring up to 70.6% fewer samples. Experiments on two autonomous racing platforms with intentional model mismatch demonstrate that SG‑RL outperforms both RL and gradient‑based policy learning baselines and generalizes zero‑shot to unseen environments.

By Baha Zarrouki, Arslan Thobani, Jasper Hoffmann, Mattia Piccinini, Rudolf Reiter, Felix Jahncke, S\'ebastien Gros, Davide Scaramuzza, Johannes Betz
arXiv AI
Sep 2

Beyond Static Summarization: Proactive Memory Extraction for LLM Agents

The paper "Beyond Static Summarization: Proactive Memory Extraction for LLM Agents" identifies two shortcomings in current memory extraction for large language model agents: (1) extraction occurs ahead of time and mixes multiple types of information, leading to loss of useful details, and (2) extraction is typically one‑off, allowing errors and hallucinations to persist. To address these issues, the authors propose ProMem, a proactive framework that separates details, events, and relations, applies distinct extraction strategies for each, checks for completeness, and verifies facts at an atomic level. Experiments demonstrate that ProMem enhances memory completeness and question‑answering accuracy while maintaining a favorable balance between quality and token cost.

By Chengyuan Yang, Zequn Sun, Wei Wei, Wei Hu
arXiv AI
Sep 2

Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning

The paper introduces the Unified Memory Agent (UMA), a system that builds a query‑agnostic external memory from a data stream and reuses it across multiple question‑answering sessions. UMA employs a single policy to manage a structured Memory Bank via CRUD operations and uses Task‑Stratified GRPO to supervise memory maintenance based on QA trajectory rewards. The authors also present Ledger‑QA, a benchmark for long‑horizon state tracking, and demonstrate that UMA outperforms other methods on test‑time learning and accurate‑retrieval tasks, with UMA‑Specialist further improving performance after task adaptation.

By Kehao Zhang, Shangtong Gui, Sheng Yang, Wei Chen, Yang Feng
arXiv Computer Vision
Sep 2

ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training

ZimaBlue is a scalable framework that learns generalizable World Action Models (WAMs) from large-scale egocentric videos. It follows a three-stage curriculum: causal video pre‑training, video‑action mid‑training with a unified action representation, and final specialization to a target robot. The system employs an asynchronous Slow‑Fast architecture to enable real‑time 30 Hz action prediction, achieving a jump in real‑robot zero‑shot success from 36.1% to 77.8% when leveraging over 120,000 hours of embodied video.

By Xionghao Wu, Yijun Yang, Shiyang Zhou, Haoze Sun, Jianhui Liu, Songsong Yu, Jiyao Zhang, Wenbo Li, Bo Wang, Guoqing Ma, Lin Song, Renjie Liao, Shenghe Zheng, Wei Tang, Xiaojuan Qi, Yanwei Li, Yuan Zhang, Zhuotao Tian, Haoyang Huang, Nan Duan
arXiv AI
Sep 2

A Hybrid Insider Threat Detection Framework Combining Multi-Agent Simulation, Layered SIEM Correlation, and Theory-of-Mind Reasoning

The paper introduces a hybrid insider threat detection framework that combines multi-agent simulation, layered SIEM correlation, trust‑adaptive thresholds, behavioral and communication forensics, and Theory‑of‑Mind reasoning. It evaluates four variants—Layered SIEM‑Core, Cognitive‑Enriched SIEM, Evidence‑Gated SIEM, and an Enron‑calibrated version—showing progressively higher actor‑level F1 scores and reduced false positives, especially with evidence gating. Domain‑shift tests reveal that an Enron‑trained email classifier does not transfer to other domains, but fine‑tuning achieves high F1, and scalability tests confirm stable performance up to 1,000 agents.

By Firdous Kausar, Asmah Muallem, Naw Safrin Sattar, Mohamed Zakaria Kurdi
arXiv AI
Sep 2

Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate

The paper introduces the Detection Surface, a geometric framework that maps the decision boundaries of heterogeneous safety filters in text‑to‑image models. Using this insight, the authors propose CRACK, a multi‑agent debate system that iteratively mutates prompts, diagnoses layer‑specific constraints, and refines attacks to bypass composite defenses. Experiments demonstrate that CRACK can achieve attack success rates up to 99.63% while using fewer queries and preserving semantic fidelity.

By Kaiyan Wen, Shijie Zhang, Lu Yu, Guangdong Bai
arXiv AI
Sep 2

SEBA: Sample-Efficient Black-Box Attacks on Visual Reinforcement Learning

SEBA is a sample‑efficient framework for black‑box adversarial attacks on visual reinforcement learning agents. It combines a shadow Q model, a generative adversarial network for imperceptible perturbations, and a world model to simulate dynamics, reducing real‑world queries. Experiments on MuJoCo and Atari show SEBA significantly lowers cumulative rewards while preserving visual fidelity and requiring far fewer environment interactions than previous methods.

By Tairan Huang, Yulin Jin, Junxu Liu, Qingqing Ye, Haibo Hu
arXiv AI
Sep 2

Keep Everyone Happy: Online Fair Division of Numerous Items with Few Copies

The paper introduces a new online fair division framework where a learner must allocate indivisible items to agents in real time, balancing fairness and efficiency. Traditional methods rely on many copies of each item to estimate utilities, but this is unrealistic for platforms with many users and few interactions. By treating utility as an unknown function of item-agent features and framing the problem as a contextual bandit, the authors propose algorithms that achieve sublinear regret and demonstrate their effectiveness experimentally.

By Arun Verma, Indrajit Saha, Makoto Yokoo, Bryan Kian Hsiang Low