arXiv Computer Vision

An Attribute-Based Measure of Video Complexity

The paper introduces Video Attribute-Based Complexity (VideoABC), a non‑parametric framework that estimates the likelihood of failure for video‑LLMs on video‑question pairs. VideoABC projects reference videos into a space defined by a pre‑defined vocabulary of video attributes (e.g., scene complexity, event speed), quantizes this space, and computes expected complexity values for each quantization cell. To handle small reference datasets, it combines a k‑means quantizer for in‑distribution samples with a universal lattice quantizer for out‑of‑distribution generalization, and uses a synthetic video generation procedure to populate lattice cells during training. Experiments demonstrate that VideoABC outperforms methods like ‘video‑LLM as judge’ while requiring fewer resources, and its attribute‑based scores provide interpretable insights into benchmark complexity.

arXiv Machine Learning
Jun 2

Perception First: A Frontier Native-Video Model with Self-Consistency for Implicit Video Question Answering

arXiv:2606. 01485v1 Announce Type: cross Abstract: We describe our submission to the VRR Challenge @ CVPR 2026, built on the \emph{ImplicitQA} / \emph{VRR-QA} benchmark~\cite{implicitqa}: multiple-choice video question answering in which answers are deliberately \emph{not} observable in any single frame and must be inferred from spatial layout, motion, depth, viewpoint, causality, and social context across discontinuous frames of creative video.

By Ali Alavi
arXiv AI
Sep 15

Evaluation of MLLM-Agnostic Plug-and-Play Keyframe Selection Methods for Long Video Understanding

The paper evaluates five training‑free, plug‑and‑play keyframe selection methods for multimodal large language models (MLLMs) on long‑video understanding tasks. It compares these methods across three different MLLMs and three video question‑answering benchmarks, finding that QAaF performs best in 13 of 15 settings while FOCUS ranks second. The study offers a unified benchmark for assessing MLLM‑agnostic keyframe selection techniques.

By Dilip Sarkar, Md. Safayet Islam, Liang Liang
arXiv AI
Sep 10

Concord: A Video Relational Algebra for Cross-Modal Query Optimization

Concord introduces a Video Relational Algebra (VRA) that models videos, transcripts, frames, and object tracks, enabling semantic video queries. It applies approximate optimizations to rewrite VRA queries, reducing large language model (MLLM) usage by processing transcripts or using detection and tracking instead of full-video MLLM joins. Experiments on soccer broadcasts and lectures show that Concord sends only a small fraction of video to the MLLM, cutting costs by up to 87%, and improves cross‑camera query accuracy from an F1 of .364 to .813 without any MLLM calls.

By Sultan Muratbek, Charisse Ivana Yeung, Chanwut Kittivorawong, Alvin Cheung
arXiv Computation and Language
3d ago

LF-VideoQA: Evaluating Long-Video Understanding with Long-Form Theory-of-Mind Questions

LF-VideoQA is a new dataset for long‑form, open‑ended question answering over narrative videos, built around theory‑of‑mind questions that require reasoning across scenes averaging 23 minutes. It contains over 6,300 question‑answer‑rubric triples generated automatically from annotated screenplays and validated by humans. The dataset challenges current vision‑language models, showing that even oracle models fall short and that visual evidence is essential for answering these questions.

By Shaden Shaar, Bradon Thymes, Majd Aldaye, Claire Cardie, Bharath Hariharan
arXiv AI
Aug 5

Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models

arXiv:2608. 03112v1 Announce Type: cross Abstract: Vision-language models excel at image and video understanding but suffer from high inference latency due to the need to process thousands of tokens per image, limiting their deployment on resource-constrained edge devices and in real-time surveillance applications.

By Paribesh Regmi, Qingshuang Chen, Chi Zhang, Heba Aly, Yelin Kim, Hongda Mao
arXiv AI
Jul 28

FilmBench: A Film-Grade Benchmark for Cinematic Video Generation

arXiv:2607. 24241v1 Announce Type: cross Abstract: Progress in video generation keeps narrowing the visual gap between AI-generated and professionally produced footage, yet most benchmarks still draw prompts from web sources or LLM templates and score them with untrained, generic multimodal models.

By Shengyi Wang, Niantong Li, Guangzheng Hu, Hong Qi, Fei Ding, Weixu Qiao, Jinlin Wang, Xiaotong Lv, Peng Han, Zimeng Li, Fanshu Ding, Yushu Wang, Han Wu, Jingjing Chen, Chongxiao Wang, Yanhao Wu, Chenglong Huang, Xiaoqian Zhu, Jie Tian, Hua Li, Jingjing Fan, Mingshuang Tang, Zhong Li, Hengxia Qiang, Weibin Chen, Jinyang Zhen, Bing Zhao, Lin Qu, Jing Li, Hu Wei
arXiv Computer Vision
Oct 15

VideoHarness-SRSI: Structured Recursive Self-Improvement for Video Understanding Harness

arXiv:2610.19424v1 Announce Type: new Abstract: LLM system capabilities depend not only on model parameters but also on the executable harnesses that organize information acquisition, tool use, reaso...

By Zihao Lin, Haoyu Dong, Haobo Xu, Chi-Chih Chang, Xuying Ning, Yinglong Xia, Shuang Yang, Zhuokai Zhao, Xinying Song, Dongfang Liu, Lifu Huang, Yang Song, Qifan Wang
arXiv Computation and Language
Sep 4

Select, Compress, Reinvest: A Controlled Study of Visual-Token Allocation in Long-Video MLLMs

The study investigates how long‑video language models decide which frames to keep, compress, and reuse, testing each decision in isolation across six selection rules, three benchmarks, and two answering models. It finds that selecting frames based on queries yields the biggest performance boost, that halving spatial resolution costs little, and that reallocating saved tokens to more compressed frames can further improve accuracy. The work also highlights the importance of a unified evaluation harness to avoid misleading comparisons.

By Prakhar Khatri