Hugging Face Trending Papers August 9, 2026

OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories

Read the original on Hugging Face Trending Papers →

Visual tool use has emerged as a fundamental capability for multimodal agents to actively acquire evidence beyond a fixed image encoding. The prevailing recipe learns this capability from teacher-generated trajectories filtered for answer correctness, implicitly assuming that every successful demonstration provides effective supervision.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Aug 11

OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories

arXiv:2608. 08557v1 Announce Type: cross Abstract: Visual tool use has emerged as a fundamental capability for multimodal agents to actively acquire evidence beyond a fixed image encoding.

By Changhao Xiang, Shilin Zhang, Zheng Ma, Kanzhi Cheng, Ruize Ma, Yi Feng, Jianbing Zhang, Zhi Wang, Zhen Wu, Xinyu Dai, Lewei Lu

agents fine-tuning multimodal benchmarks

arXiv AI

4d ago

On-Policy Visual Evidence Distillation

arXiv:2609.36838v1 Announce Type: cross Abstract: Visual agents solve problems by interleaving reasoning with image operations, and on-policy distillation (OPD) provides guidance from a strong teache...

By Shaohang Wei, Feifan Song, Guangyue Peng, Wenhao Yu, Wei Li, Wen Luo, Yang Xu, Yufan Shen, Luke Mao, Yang Du, Asher Qin, Houfeng Wang

agents efficiency multimodal benchmarks

arXiv AI

Aug 11

ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervision

arXiv:2608. 08907v1 Announce Type: cross Abstract: Thinking with images allows a multimodal model to compensate for limited perception by invoking visual tools through code.

By Delin Mao, Chenghao Sun, Jingwei Song, Chishui Chen, Linfeng Zhang

agents reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jun 19

Vero: An Open RL Recipe for General Visual Reasoning

arXiv:2604. 04917v3 Announce Type: replace-cross Abstract: What does it take to build a visual reasoner that works across charts, science, spatial understanding, and open-ended tasks?

By Gabriel Sarch, Linrong Cai, Qunzhong Wang, Haoyang Wu, Danqi Chen, Zhuang Liu

llms reinforcement-learning efficiency multimodal benchmarks

arXiv AI

Sep 4

Making Every Tool Call Count: Necessary Tool-Evidence Path Rewards for Agentic Vision-Language Models

The paper introduces the Necessary Tool‑Evidence Path (NTEP) annotation scheme and its associated reward mechanism (NTEP‑R) to better supervise vision‑language models that use external tools. By explicitly specifying which evidence is needed and penalizing redundant tool calls, the authors train an 8B‑parameter model that shows improved accuracy and tool‑use efficiency across seven image‑grounded benchmarks. The approach demonstrates that fine‑grained supervision of tool‑evidence paths is essential for robust agentic VLM performance.

By Xingming Long, Yu Liu, Zhiwei Yang, Hanqi Feng, Shaojie Zhang, Barnabas Poczos, Chao Jiang, Zhenbo Luo, Lei Jiang, Pei Fu

llms agents multimodal benchmarks

arXiv AI

Jul 2

XSkill: Continual Learning from Experience and Skills in Multimodal Agents

arXiv:2603. 12056v3 Announce Type: replace Abstract: Multimodal agents can now tackle complex reasoning tasks with diverse tools, yet they still suffer from inefficient tool use and inflexible orchestration in open-ended settings.

By Guanyu Jiang, Zhaochen Su, Xiaoye Qu, Yi R. Fung

agents nlp multimodal benchmarks

One email a morning, machine-written

Related stories

OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories

On-Policy Visual Evidence Distillation

ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervision

Vero: An Open RL Recipe for General Visual Reasoning

Making Every Tool Call Count: Necessary Tool-Evidence Path Rewards for Agentic Vision-Language Models

XSkill: Continual Learning from Experience and Skills in Multimodal Agents