VideoLVR introduces a method for multimodal large language models to maintain visual grounding during chain-of-thought reasoning by constructing latent visual evidence from local video regions at relevant moments. It uses a visual token prediction objective that unifies visual evidence reconstruction and text generation, allowing models to interleave text tokens with latent visual states. The approach is further extended with reinforcement learning, treating visual tokens as actions and optimizing policies with final-answer rewards, thereby improving video understanding without expanding the multimodal context.
By Shilong Li, Xingyuan Bu, Hao Shi, Yun Xing, Weixin Feng, Qiangpeng Yang, Shilei Wen
The paper introduces the Very Big Video Reasoning (VBVR) Dataset, a large-scale collection of over one million video clips organized into 200 curated reasoning tasks. It also presents VBVR-Bench, a benchmark framework that uses rule-based, human-aligned scorers for reproducible evaluation of video reasoning models. The authors conduct a large-scale scaling study, noting early signs of emergent generalization to unseen reasoning tasks, and make all resources publicly available.
By Maijunxian Wang, Ruisi Wang, Juyi Lin, Ran Ji, Thadd\"aus Wiedemer, Qingying Gao, Dezhi Luo, Yaoyao Qian, Lianyu Huang, Zelong Hong, Jiahui Ge, Qianli Ma, Hang He, Yifan Zhou, Lingzi Guo, Lantao Mei, Jiachen Li, Hanwen Xing, Tianqi Zhao, Fengyuan Yu, Weihang Xiao, Yizheng Jiao, Jianheng Hou, Danyang Zhang, Pengcheng Xu, Boyang Zhong, Zehong Zhao, Gaoyun Fang, John Kitaoka, Yile Xu, Hua Xu, Kenton Blacutt, Tin Nguyen, Siyuan Song, Haoran Sun, Shaoyue Wen, Linyang He, Runming Wang, Yanzhi Wang, Mengyue Yang, Ziqiao Ma, Rapha\"el Milli\`ere, Freda Shi, Nuno Vasconcelos, Daniel Khashabi, Alan Yuille, Yilun Du, Ziming Liu, Bo Li, Dahua Lin, Ziwei Liu, Vikash Kumar, Yijiang Li, Lei Yang, Zhongang Cai, Hokin Deng
arXiv:2608. 15869v1 Announce Type: cross Abstract: Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments.
By Xiaoyu Zhu, Xinke Deng, Suresh Taddewadikar, Arnab Kumar Mondal, Zhongyu Jiang, Ian Fasel, Joerg Liebelt