AI agents

Tool use, function calling, orchestration and the protocols that let models act rather than only answer.

15,576 stories · RSS feed

arXiv Computer Vision
1d ago

ImagineAgent: What if Agents Could Imagine? Reinforcing Open-Vocabulary HOI Comprehension through Generation

arXiv:2602.11499v3 Announce Type: replace Abstract: Multimodal Large Language Models have shown promising capabilities in bridging visual and textual reasoning, yet their reasoning capabilities in Op...

By Zhenlong Yuan, Yue Wang, Dapeng Zhang, Kejin Cui, Rui Chen, Jing Tang, Lei Sun, Hongwei Yu, Chengxuan Qian, Xiangxiang Chu, Shuo Li, Yuyin Zhou
arXiv Computer Vision
1d ago

ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning

arXiv:2605.20342v3 Announce Type: replace Abstract: Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e.g., cropping) has become a pro...

By Zuhao Yang, Kaichen Zhang, Sudong Wang, Keming Wu, Zhongyu Yang, Bo Li, Xiaojuan Qi, Shijian Lu, Xingxuan Li, Lidong Bing
arXiv Computer Vision
1d ago

Learning to Live: A Step Toward Lifelong Embodied Intelligence

arXiv:2611.00180v2 Announce Type: replace-cross Abstract: Embodied agents are usually organized around individual tasks, yet the physical world keeps changing between tasks and during execution. We i...

By Lirong Che, Peidong Wang, Yimeng Su, Jiong Wang, Yang Chen, Wenbo Fu, Zhenyu Huang, Tiancheng Xu, Yuzhe Yang, Dongli Xie, Xu Cao, Chuang Wang, Daqi Liu, Jian Su
arXiv AI
1d ago

Iterated creation, grading, and revision of data science projects by language models

The study evaluates how agentic AI models can autonomously complete end‑to‑end data science projects and improve through iterative feedback. A writer model generates a full project—from data acquisition to analysis and paper writing—while a grader model evaluates it; this cycle repeats for up to three rounds. Across 49 writer‑grader pairs from five developers, initial submissions improved from an average of 59 % to 76 % after feedback, with variations depending on the specific model roles.

By Rohan Alexander
arXiv AI
1d ago

PAYWORTHY: Can LLMs Predict Which Freelance Jobs an AI Agent Will Complete?

PAYWORTHY is a new benchmark that tests whether large language models can predict the success of an AI agent on freelance jobs before the agent attempts them. The dataset contains 1,494 real job postings sampled from 18.8 million postings on a freelancing platform, and 18 models are asked to forecast the likelihood of success. When the agent’s limitations are explicitly described, models achieve AUC scores between 0.66 and 0.86; when the agent is described generically, the models perform near random (AUC 0.45–0.56).

By Mohammad Reza Rezaei, Dingxian Wang, Brett Levert, Celeste Fejzo, Andrew Rabinovich, Zhao Chen
arXiv AI
1d ago

Autonomous agents for electricity-system planning

The paper introduces an autonomous planning framework for electricity systems that iteratively proposes, evaluates, and refines investment and operational decisions across regional scenarios. Agents generate investment bounds, storage conditions, and feeder interventions, interpret numerical feedback, and decide when to stop, automatically running annual investment-and-dispatch optimization and feeder verification. The framework was tested on forty carbon‑constrained trajectories across five systems and four countries, successfully retaining low‑cost candidates that meet physical and carbon‑budget constraints, with 28 of 40 matching or exceeding reference designs.

By Yoon Pyo Lee, Samrendra Roy, Jay Phil Yoo, Kazuma Kobayashi, Souvik Chakraborty, Syed Bahauddin Alam
arXiv AI
1d ago

ModalSearcher: Weaving Visual Evidence into Autonomous Cross-Modal Deep Search

ModalSearcher introduces an autonomous cross‑modal search framework that lets an agent dynamically choose among direct reasoning, text‑to‑text search, text‑to‑image search, and reverse‑image search, actively incorporating visual evidence as needed. The system builds an interleaved multimodal knowledge graph via a scalable pipeline that samples diverse paths and uses backward question construction to prevent shortcut reasoning. Trained with supervised fine‑tuning and reinforcement learning, ModalSearcher‑8B outperforms Qwen3‑VL‑8B by 16.2 accuracy points across five multimodal knowledge‑intensive QA benchmarks and shows consistent gains on active visual‑search tasks.

By Zhihan Yin, Ruyi Li, Chengyu Fang, Yifeng Yao, Nonghai Zhang, Pengfei Xiong, Terry Zhang, Huishuai Zhang, Dongyan Zhao
arXiv AI
1d ago

Time-Division Multiplexing: The Operational Rhythm and Temporal Organization Layer of Digital Life

The paper proposes a temporal organization layer for continual agents, introducing phase‑tagged micro‑epochs, a duty cycle, and nested clocks to separate exploration from commitment. It presents deterministic and stochastic guarantees for duty‑cycle updates, defines busy/idle execution contracts, and evaluates the design through synthetic experiments and a Qwen3‑8B prototype. The results demonstrate a testable scheduling design and clarify its conditions of validity.

By Haibin Tong, Luming Guo, Nana Cui, Jiang Yu
arXiv AI
1d ago

Do agents need to sleep?

The paper argues that large language model agents, which persist across sessions and accumulate memory, suffer from unchecked plasticity leading to semantic drift, retrieval errors, and memory saturation. It draws on a selectionist theory of sleep—comprising ringing, resonance, and renormalization—to propose that each phase can be implemented in digital memory systems. The authors predict that selectionist consolidation will outperform instructionist methods as memory stores grow, suggesting that sleep is a necessary cost of continuous learning under finite capacity.

By Ido Haber, Giulio Tononi
arXiv AI
1d ago

BobFlow: Towards Human-Agent Collaborative Data Science Using Dataflows

BobFlow proposes using dataflows as a shared computational abstraction to facilitate human-agent collaboration in data science, combining this with the ReAct paradigm to allow agents to iteratively build and run dataflows while analysts can inspect and provide feedback. The system demonstrates that analysts can better understand, inspect, and verify agent-generated analyses, and experimental results show accuracy gains of up to 18.6 points at roughly one‑third the inference cost of a script‑based ReAct agent.

By Jiadong Bai, Yicong Huang, Chen Li
arXiv AI
1d ago

GREP: Grounding Generated Room Evaluation in Principles

The paper introduces GREP, a framework for evaluating AI-generated furnished rooms that moves beyond specific questions to a three-tier philosophy: basic physical validity, visual coherence, and human-use logic. An autonomous evaluation agent applies these principles to identify scene issues, produce scores, and generate reports, enabling reliable, scene-specific judgments. Experiments demonstrate GREP’s high agreement with human preferences, its ability to improve scene quality through feedback, and insights into how future models might better balance planning and constraint adaptation.

By Mohan Cao, Yongsen Mao, Chucheng Xiang, Zhongyuan Liu, Zhihang Zhong, Haiping Wang, Yuan Liu
arXiv AI
1d ago

Autonomous ETL Pipelines for Salesforce M&A Consolidation: A Reference Framework, Jitterbit Mapping, and Reproducible Simulation Study

The article presents a reference framework for autonomous extract‑transform‑load (ETL) pipelines to consolidate Salesforce data during mergers and acquisitions. It details a partitioned bulk extraction, declarative mapping, rule‑based and statistical quality gates, master data management with deterministic and blocked fuzzy matching, constrained loading with error capture, and change data capture protected by a data‑health circuit breaker. The prototype was evaluated on synthetic scenarios, achieving high F1 scores for account and contact matching and preventing load failures and data loss.

By Imran Basha Shaik, Sahla Sultana, Khader Moinuddin Shaik
arXiv AI
1d ago

Zero-Shot Coordination among LLM Agents

The paper investigates zero‑shot coordination (ZSC) among large language model (LLM) agents, a scenario where independently developed agents must coordinate without prior shared scaffolding. Experiments in minimal, controlled environments reveal that current frontier LLMs struggle to coordinate, mainly due to limited understanding of the coordination problem and weak reasoning about partners’ beliefs. The authors introduce Coordination‑Friendly Definitions (CFDs) as a principled, automatically discoverable approach to improve robust coordination among LLM agents.

By Adrian Hayler, Shashank Reddy Chirra, Andrei Lupu, Johannes Forkel, Bidipta Sarkar, Siheng Feng, Jakob Foerster
arXiv AI
1d ago

EvoWorld: Towards Out-of-Sight World Evolution in Generative Video Models

EvoWorld introduces a framework that couples a vision‑language model with a segment‑autoregressive video renderer to enable generative video models to reason about world evolution beyond visual fidelity. The system maintains an evolving account of the world, infers event consequences, and renders causally plausible outcomes even when events occur out of sight. The accompanying EvoWorld‑Data and EvoWorld‑Bench datasets provide training and evaluation for event realization, causal reactions, and revisit consistency.

By Yequan Bie, Dengyang Jiang, Zhixuan Chen, Xiaozhuan Liang, Haoxuan Che, Quande Liu, Xi Chen, Hao Chen
arXiv AI
1d ago

When the Agent Learns Too: Learning to Incentivize with Unobserved Rewards

The paper studies a repeated principal–agent game where the principal cannot observe the agent’s rewards and both parties learn over time. It introduces a non‑parametric estimator that uses only the history of incentives and choices, and combines it with a data‑driven incentive policy in a multi‑armed bandit setting. The authors prove finite‑sample consistency of the estimator and provide a regret bound for the principal, supported by simulations on green‑energy aggregator contracts.

By Ilgin Dogan, Zuo-Jun Max Shen, Anil Aswani
arXiv Computation and Language
1d ago

Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents

The paper introduces a visual-native agent harness that registers every tool‑returned image as an addressable reference, enabling reuse of intermediate visual evidence in multimodal deep search tasks. It also proposes On‑Policy Data Evolution (ODE), a closed‑loop data generator that refines training data across rounds based on policy rollouts, targeting the agent’s evolving needs. Experiments on eight benchmarks show ODE boosts the Qwen3‑VL‑8B agent’s performance from 24.9% to 39.0% on average, and from 30.6% to 41.5% at 30B, while validating the benefits of image bank reuse and rollout‑feedback evolution.

By Shijue Huang, Hangyu Guo, Guanting Dong, Chenxin Li, Junting Lu, Xinyu Geng, Zhaochen Su, Zhenyu Li, Shuang Chen, Hongru Wang, Yi R. Fung
arXiv AI
1d ago

SlackBench: Benchmarking Agents on Collaborative Projects Grounded in Real Code Repositories

SlackBench is a new benchmark that evaluates AI agents on realistic collaborative research and engineering workflows, requiring joint reasoning over workplace communication and evolving repository state. The benchmark is built from privacy‑preserving reenactments of real academic projects, providing reproducible environments with multi‑party dialogue, code changes from GitHub, and Google Docs. It contains 140 queries covering heterogeneous‑source search, abstention, summarization, and inference over long conversations, and shows that current models perform well below the ceiling, indicating significant room for improvement.

By Mingxuan Li, Huanzhi Mao, Ruitao Zheng, Jarod Stanbury James, Yunze Li, Natanya Anderson, Joseph E. Gonzalez, Alex Dimakis