arXiv Computer Vision
Aug 27

Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios

Video-IFBench is a new benchmark designed to evaluate how well multimodal large language models (MLLMs) follow user-specified instructions in video understanding tasks. It introduces an instruction taxonomy with four templates—single-task, multi-task, selection, and nested—covering 32 task types and 39 constraint categories that span semantic and format requirements. The benchmark was built using a semi-automatic pipeline that combines MLLMs, programmatic processing, and human verification, producing 1.5K samples, and a large-scale evaluation of over 20 recent MLLMs shows that instruction following remains difficult, especially for complex constraints and conditional structures.

By Hongbo Liu, Peixian Chen, Sihan Liu, Peiyuan Zhang, Kai Zou, Dian Zheng, Xiaoxing Hu, Yuhao Dong, Mengdan Zhang, Yunhang Shen, Haoyu Cao, Wei Liu, Weibo Gu, Xing Sun, Shengjie Zhao
arXiv Computation and Language
Aug 31

MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answering

MemoryCard is a video-memory-based augmentation framework designed to improve long-video question answering for Vision‑Language Models. It segments lengthy videos into semantically coherent units—each representing a distinct topic or event—by performing a self‑reading process over the video and aligned utterances. For each unit, the framework generates an event‑level video gist and selects representative visual moments, which are compiled into unified Memory Cards that are used for retrieval and answering questions, yielding up to a 21.8% relative accuracy improvement under comparable visual‑token budgets.

By Qing Yang, Pengcheng Huang, Xinze Li, Zhenghao Liu, Yukun Yan, Yu Gu, Ge Yu, Gang Li, Maosong Sun