The paper introduces AllocEmbed, an allocate‑then‑embed framework that reallocates a fixed visual‑input budget across more video frames to improve retrieval performance. A lightweight allocator uses low‑cost previews to assign frame‑wise resolutions before the embedding backbone, preserving detail where it most benefits retrieval while reducing visual cost elsewhere. Retrieval‑Driven Policy Optimization (RDPO) learns the allocator directly from retrieval feedback, and the method integrates with existing systems without modifying the embedding model.
By Song Jin, Zhongtao Jiang, Chenglei Shen, Huanxuan Liao, Haozhe Chi, Zhiwei Wang, Kun Xu, Yong Liu
arXiv:2609.37042v1 Announce Type: cross
Abstract: Video Large Language Models (VideoLLMs) have achieved strong video understanding capabilities but incur substantial inference overhead due to the lar...
By Shuo Yang, Changbai Li, Rui Tang, Xinyu Zhao, Linlin Yang, Baochang Zhang
arXiv:2608. 13990v1 Announce Type: new Abstract: Driven by the attention economy, short-video Recommender Systems (RSs) are primarily optimized to maximize user engagement by promoting videos that capture attention within seconds.
By Liwei Deng, Jing Jiang, Zhiwei Li, Yang Wang, Guodong Long
arXiv:2608.05707v2 Announce Type: replace
Abstract: Frame selection is essential for applying Large Multimodal Models (LMMs) to long videos due to severe frame redundancy and limited context windows....
By Wang Chen, Yu Chen, Xiang Wang, Shuai Li, Jinfa Huang, Xiawu Zheng
Most keyframe selection studies focus on offline settings, assuming access to the full video and query in advance. In contrast, real-world streaming scenarios require online frame selection under unkn...
SVMemAgent introduces a streaming video memory (SVMem) that continuously updates a compact representation of observed frames for online keyframe selection without prior knowledge of video length, query, or future frames. The agent decides at each timestep whether to replace an existing memory frame with a new one or discard it, trained via Group Relative Policy Optimization using task-driven rewards from question-answer pairs. Experiments demonstrate that SVMemAgent outperforms existing online baselines and rivals offline methods, and its learned policy tends to favor frames containing textual information, potentially aiding downstream VideoQA tasks.
By Dohwan Ko, Ji Soo Lee, Pierce Chuang, Debojeet Chatterjee, Ashish Shenoy, Yichao Lu, Seungwhan Moon, Xin Luna Dong, Vikas Bhardwaj, Hyunwoo J. Kim