arXiv Computer Vision

FaithEyes: Towards Faithful Tool Use via Multi-Agent Process-Image Self-Verification

arXiv Computer Vision
Sep 30

Beacon: Knowing When and How to Perform Agentic Visual Reasoning

Beacon is a new agentic visual reasoning model that improves multimodal large language models (MLLMs) by better deciding when to use tools and how to use them. It introduces two key concepts—Mode Adaptiveness, which ensures tools are invoked only when necessary, and Tool Effect, which measures the net benefit of tool use— and trains the model with supervised fine‑tuning and reinforcement learning that rewards necessity-aware decisions and expands capability through expert hints. Across 13 benchmarks, Beacon outperforms other open‑source models, achieving the highest average score and the largest net tool‑gain on diagnostic tests.

By Qixun Wang, Yang Shi, Letian Cheng, Zhuoran Zhang, Yan He, Yuqi Tang, Qi Zhang, Xinlei Yu, Ruizhe Chen, Tianrun Xu, Yuanxing Zhang, Pengfei Wan, Haotian Wang, Xianghua Ying
arXiv AI
Sep 4

Making Every Tool Call Count: Necessary Tool-Evidence Path Rewards for Agentic Vision-Language Models

The paper introduces the Necessary Tool‑Evidence Path (NTEP) annotation scheme and its associated reward mechanism (NTEP‑R) to better supervise vision‑language models that use external tools. By explicitly specifying which evidence is needed and penalizing redundant tool calls, the authors train an 8B‑parameter model that shows improved accuracy and tool‑use efficiency across seven image‑grounded benchmarks. The approach demonstrates that fine‑grained supervision of tool‑evidence paths is essential for robust agentic VLM performance.

By Xingming Long, Yu Liu, Zhiwei Yang, Hanqi Feng, Shaojie Zhang, Barnabas Poczos, Chao Jiang, Zhenbo Luo, Lei Jiang, Pei Fu
arXiv AI
6d ago

With Tools or Because of Tools? Tracing Capability Gains in Multimodal Agents

The paper examines whether tool-augmented multimodal agents truly gain capability from using tools. By comparing agents Thyme and DeepEyesV2 with tool-free and pure-text counterparts across tasks such as image understanding, OCR, chart reading, and math reasoning, the authors find that tool access offers little consistent improvement and that most problems solved with tools are also solvable without them. Ablation studies reveal that the full tool-use loop does not consistently outperform simpler tool-call or result-only formats, indicating that agents may learn tool-calling patterns rather than tool-contributed capabilities.

By Jiawei Guo, Donglei Yu, Yu Chen, Xiang Wang, Shuai Li, Xinpei Zhao, Huaxing Liu, Qinghao Wang, Minpeng Liao
arXiv AI
Oct 7

When to Rethink: Learning Multi-Perspective Self-Verification for Vision-Language Models

The paper introduces MOTIVE, a multi-view self-verification framework for vision-language models that evaluates candidate answers from complementary perspectives and learns a reliability score aligned with correctness. By using this score to decide whether to accept an answer or trigger history-guided rethinking, MOTIVE improves reliability and efficiency compared to existing self-verification and self-correction methods. Extensive experiments on diverse multimodal benchmarks and VLM backbones show consistent performance gains and reduced unnecessary reasoning turns.

By Ziquan Zhu, Hanruo Zhu, Si-Yuan Lu, Morris Yu-Chao Huang, Yicheng Lin, Wei Han, Tianlong Chen, Mingyuan Wu, Hanchao Yu, Gaojie Jin, Lu Liu, Bo Sun, Tianjin Huang
arXiv AI
Oct 16

CoWorld Reasoner: Agentic Multimodal Reasoning with the Evolving Video World Model-Driven Harness

CoWorld Reasoner is an agentic framework that leverages a Video World Model (VWM) as a unified visual reasoning engine for Vision‑Language Models (VLMs). It exposes multiple visual operations through tools and skills, allowing the VLM to decide when to imagine, select an operation, generate visual content, and interpret the resulting evidence. The framework includes a VLM‑VWM Harness Co‑Evolution mechanism that jointly refines tools, skills, and execution code based on feedback, while keeping model weights fixed. Across seven benchmarks covering spatial, temporal, causal, and perceptual reasoning, CoWorld Reasoner outperforms direct VLM reasoning and improves accuracy on all benchmarks, achieving notable gains on MMSI and ERQA.

By Dongjie Cheng, Ruifeng Yuan, Ran Wei, Ning Song, Yongqi Li, Wenjie Li
arXiv AI
1d ago

Vision-as-a-Tool: Visual Harness for Multimodal Agentic Coding

The paper introduces Vision-as-a-Tool (VaaT), a paradigm that treats multimodal large language models as external utilities for visual reasoning, allowing text-only agents to offload visual tasks and vision-capable agents to augment perception on demand. It presents VHarness, a unified framework with five atomic actions, and evaluates it on Image-to-HTML, Image-to-SVG, and Image-to-3D generation tasks. Results show consistent improvements in visual fidelity and geometric accuracy for both text-only and multimodal coding agents.

By Haolin Yang, Wenshan Wu, Jin Jiang, Yu Cheng, Yan Xia, Ting Song, Yangyu Huang, Zhiliang Peng, Wenhui Wang, Hangbo Bao, Yuzhe Zhang, Wen-Da Wei, Tengchao Lv, Li Dong, Lei Cui, Furu Wei
arXiv AI
Aug 25

ATP-Bench: Towards Agentic Tool Planning for MLLM Interleaved Generation

ATP‑Bench proposes a new benchmark for evaluating agentic tool planning in multimodal large language models (MLLMs) that generate interleaved text-and-image responses. The benchmark contains 7,702 QA pairs, including 1,592 visual‑question‑answer pairs, across eight categories and 25 visual‑critical intents, all verified by humans. A Multi‑Agent MLLM‑as‑a‑Judge (MAM) system is introduced to assess tool‑call precision, missed opportunities, and overall response quality without relying on ground‑truth references.

By Yinuo Liu, Zi Qian, Heng Zhou, Jiahao Zhang, Yajie Zhang, Zhihang Li, Mengyu Zhou, Erchao Zhao, Xiaoxi Jiang, Guanjun Jiang
arXiv Machine Learning
Jul 31

LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger

arXiv:2607. 28374v1 Announce Type: new Abstract: Multimodal agents for visual question answering increasingly operate as multi-step trajectories that interleave perception, retrieval, and reasoning, yet evaluation still largely reduces to final-answer accuracy.

By Enjun Du, Hange Zhou, Chenxu Du, Siyi Liu, Zirong Chen, Ziyu Zheng, Yongqi Zhang