arXiv AI By Xu Xu (Beihang University), Jinxiu Liu (The Chinese University of Hong Kong), Zhangbo Qiao (Beihang University), Jiaxing Lu (Beihang University), Xiangyu Zhang (Beihang University), Yubin Gu (National University of Singapore), Fangwei Ning (Beihang University), Yan Shi (Beihang University)

OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning

Read the original on arXiv AI →

OmniHarness is a framework that enables generalizable visual generation by learning symbolic policies from verified executions. It abstracts shared procedures and applicability conditions, allowing these policies to be instantiated, adapted, and composed for new tasks while keeping model parameters fixed. The system uses intermediate verification for refinement, self-directed inquiry to generate practice tasks, and continuous feedback to expand capabilities, achieving strong results on multiple benchmarks and outperforming baselines on Creative tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
1d ago

Vision-as-a-Tool: Visual Harness for Multimodal Agentic Coding

The paper introduces Vision-as-a-Tool (VaaT), a paradigm that treats multimodal large language models as external utilities for visual reasoning, allowing text-only agents to offload visual tasks and vision-capable agents to augment perception on demand. It presents VHarness, a unified framework with five atomic actions, and evaluates it on Image-to-HTML, Image-to-SVG, and Image-to-3D generation tasks. Results show consistent improvements in visual fidelity and geometric accuracy for both text-only and multimodal coding agents.

By Haolin Yang, Wenshan Wu, Jin Jiang, Yu Cheng, Yan Xia, Ting Song, Yangyu Huang, Zhiliang Peng, Wenhui Wang, Hangbo Bao, Yuzhe Zhang, Wen-Da Wei, Tengchao Lv, Li Dong, Lei Cui, Furu Wei
arXiv Computer Vision
Sep 30

Beacon: Knowing When and How to Perform Agentic Visual Reasoning

Beacon is a new agentic visual reasoning model that improves multimodal large language models (MLLMs) by better deciding when to use tools and how to use them. It introduces two key concepts—Mode Adaptiveness, which ensures tools are invoked only when necessary, and Tool Effect, which measures the net benefit of tool use— and trains the model with supervised fine‑tuning and reinforcement learning that rewards necessity-aware decisions and expands capability through expert hints. Across 13 benchmarks, Beacon outperforms other open‑source models, achieving the highest average score and the largest net tool‑gain on diagnostic tests.

By Qixun Wang, Yang Shi, Letian Cheng, Zhuoran Zhang, Yan He, Yuqi Tang, Qi Zhang, Xinlei Yu, Ruizhe Chen, Tianrun Xu, Yuanxing Zhang, Pengfei Wan, Haotian Wang, Xianghua Ying
arXiv Computation and Language
Aug 27

VISA: Agentic Self-Evolving Data Synthesis for Multimodal Instruction Following

VISA (Visual Instruction Synthesis Agent) is an agentic framework that transforms multimodal instruction synthesis into a self‑evolving loop. Each cycle analyzes images to filter constraints, samples new constraint sets, generates candidate instructions, and verifies them using executable tools and large language model judges. Failed samples trigger diagnostic recovery, while accepted samples are evaluated against the target model to estimate difficulty, with all feedback written back to memory to adapt future rounds and provide reward signals for reinforcement learning.

By Min Zeng, Guanxin Tan, Libin Cen, Yawei Wen, Rui Hu, Liuyang Bian, Xiaolong Chen, Xiaoxin Chen
arXiv Computer Vision
Sep 2

Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System

The paper investigates how visual understanding and generation objectives interact within unified multimodal models (UMMs). At the representation level, each objective enriches the other, but forcing them through the same computation path can cause one to dominate; a task‑decoupled architecture mitigates this. At the task and system levels, the authors demonstrate bidirectional transfer between shared knowledge and superior performance of an end‑to‑end UMM over a planner–executor pipeline on complex tasks.

By Penghao Wu, Haiwen Diao, Weichen Fan, Lewei Lu, Dahua Lin, Ziwei Liu