arXiv Computer Vision By Zixiao Gu, Yabo Chen, Xunzhi Xiang, Yu He, Haibin Huang, Chi Zhang, Yunbo Wang, Xuelong Li

Search-to-World: Evaluation of 3D World Delivery from User Request through Web Search

Read the original on arXiv Computer Vision →

The paper introduces Search-to-World, an end‑to‑end evaluation task that measures how well agentic systems can turn user requests into usable 3D worlds by combining request understanding, web visual‑content retrieval, and world delivery. It defines two metrics—Observed Retrieval Rate (ORR) and World Delivery Rate (WDR)—to separate the act of finding relevant content from successfully delivering a perceptually acceptable 3D world. The authors also present WorldSearcher, a reusable‑then‑reconstruction harness that connects existing search agents to world delivery, and a structured recovery controller that can revise queries or source videos when failures occur. Benchmarking with WorldSearcher shows that delivery success varies with the underlying agentic model and that observing relevant content alone does not guarantee world delivery; jointly training recovery agents improves both success and efficiency.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI
Sep 1

WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents

WeAgent-MMSearch introduces a multimodal search agent that preserves retrieved images as persistent references, enabling the model to inspect, process, and cite them throughout a search trajectory. The system includes a harness (WeAgent-Harness), a post‑training method (FA‑GSPO) that recovers salvageable rollouts, and a new benchmark (VisTarget‑Bench) to evaluate image‑retrieval versus visual‑perception failures. Evaluation shows that agentic post‑training boosts performance by 19.22 points, allowing the model to outperform similarly sized open‑source models and compete with much larger ones.

By Zongkai Liu, Hui Zhang, Liqiang Niu, Zhen Cao, Han Li, Juntao Liu, Wenchao Chen, Chengduo Zhao, Chao Yu, Fandong Meng
arXiv AI
Sep 1

RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction

RoboPhys-3D is a 3D‑grounded embodied world model benchmark built on RoboTwin 2.0, featuring 50 manipulation tasks, 5,000 episodes, and 25,000 multi‑view ground‑truth videos. It evaluates video world models by processing both generated and ground‑truth videos through the same 3D reconstruction pipeline, allowing the separation of reconstruction‑induced from generation‑induced errors. The benchmark defines 50 metrics across four sub‑dimensions—pixel fidelity, 3D geometry consistency, state understanding, and task completeness—and introduces the Average Full Score and RoboPhyscore for holistic assessment, with RoboPhyscore showing strong correlation with human judgments.

By Tianyi Wang, Jiazhou Chen, Yiming Xu, Xiangyu Li, Tianyi Zeng, Chih-Hsien Chou, Ning Lu, Liang Peng, Junfeng Jiao, Christian Claudel
Hugging Face Trending Papers
Oct 29

WorldProbe: When Video World Models Fall Apart under Long-Horizon Spatial Exploration

WorldProbe is a new diagnostic benchmark that tests video world models by letting a goal‑conditioned multimodal agent adapt camera actions online during generation, creating 164 test cases across four probing families. The benchmark evaluates 10 state‑of‑the‑art models over more than 1,600 rollouts, totaling roughly 800 minutes and horizons up to 180 seconds, revealing that while local visual and content quality can be maintained for longer, long‑range memory and evolution remain weak. The study highlights a gap between locally stable generation and persistent world consistency that existing short‑interaction benchmarks miss.

arXiv Computer Vision
Sep 15

BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender

arXiv:2609.15478v1 Announce Type: new Abstract: Multimodal agents can create complex videos in software such as Blender by coding without relying on diffusion models. Yet video understanding benchmar...

By Yolo Y. Tang, Daiki Shimada, Jiayue Meng, Jing Bi, Pinxin Liu, Yicheng Wang, Yunzhong Xiao, Zhangyun Tan, Zeliang Zhang, Chao Huang, Susan Liang, Qianxiang Shen, Luchuan Song, Ali Vosoughi, Mingqian Feng, Melika Filvantorkaman, Chenliang Xu
arXiv AI
Aug 5

Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent

arXiv:2608. 03979v1 Announce Type: cross Abstract: We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding coupled with open-web exploration.

By Zhen Fang, Yu Zeng, Wenxuan Huang, Yiming Zhao, Shiting Huang, Tianfei Ren, Qi Lu, Qingnan Ren, Qisheng Su, Lionel Z. Wang, Qingyu Yin, Shuang Chen, Zehui Chen, Lin Chen, Zhenfei Yin, Yao Hu, Shaohui Lin, Wanli Ouyang, Shaosheng Cao, Feng Zhao