The study investigates how long‑video language models decide which frames to keep, compress, and reuse, testing each decision in isolation across six selection rules, three benchmarks, and two answering models. It finds that selecting frames based on queries yields the biggest performance boost, that halving spatial resolution costs little, and that reallocating saved tokens to more compressed frames can further improve accuracy. The work also highlights the importance of a unified evaluation harness to avoid misleading comparisons.
By Prakhar Khatri
The paper investigates how the choice of retrieval encoder affects cache‑based test‑time adaptation for CLIP. By keeping the memory fixed and varying the retrieval space across sixteen encoders, the authors show that retrieval space can dramatically alter performance, with gains ranging from +0.44 to +19.7 points on ImageNet‑A. They introduce MARC, a training‑free system that pairs frozen CLIP with DINOv2‑B for retrieval, achieving superior out‑of‑distribution accuracy and efficiency compared to prior methods.
By Mahir Shahriar Tamim, Md. Samiul Alim, Azmine Toushik Wasi, Shahriyar Zaman Ridoy, Meharun Nesa, Mohammad Abu Yousuf, Alex Lamb, Mohammad Ali Moni
Cache-based test-time adaptation improves CLIP predictions by storing and retrieving examples from the target stream while keeping the model frozen. However, existing methods largely treat the feature...
arXiv:2606. 13141v1 Announce Type: new Abstract: Retrieval-augmented generation is moving beyond text into long, egocentric video, where systems must select query-relevant chunks across multiple modalities and temporal granularities.
By Yuho Lee, Jisu Shin, Nicole Hee-Yeon Kim, Jihwan Bang, Juntae Lee, Kyuwoong Hwang, Fatih Porikli, Hwanjun Song
arXiv:2609.39096v1 Announce Type: new
Abstract: Autoregressive video diffusion supports streaming generation and interactive control, but its KV cache grows with the generated history. Existing compr...
By Zeqi Xiao, Qingle Liu, Kaiwen Zhang, Yifan Zhou, Zihan Ding, Xingang Pan
The paper introduces RECAP, a method that improves speculative decoding for vision‑language models by recycling a target model’s own predictions. RECAP stores predictions using one‑, two‑, and three‑token keys, allowing contextual memories that are request‑specific and token‑level memories that persist across requests. By fusing retrieved predictions based on entropy‑confidence weighting and using best‑first search to build a candidate tree, RECAP achieves significant speedups over autoregressive decoding without requiring training or a draft model.
By Hengjie Zhu, Dayan Wu, Xinze Liu, Peng Fu, Zheng Lin, Weiping Wang
The paper introduces Growing Sparsity and Reduced Search (GSRS), a training‑free framework that accelerates Diffusion Transformers for video generation by progressively increasing sparsity and reusing early attention blocks to limit search space. GSRS uses Top‑p selection for adaptive sparsity and a precise evaluation scheme that computes exact attention weights with negligible overhead. Experiments on HunyuanVideo and Wan 2.1 show 1.55×–2.21× speedups while preserving generation quality comparable to full attention.
By Yun Dai, Yuyun Chen, Jiarui Wen, Huiping Zhuang, Cen Chen, Ziqian Zeng
arXiv:2609.22562v1 Announce Type: new
Abstract: Multi-vector visual document retrieval (VDR) models such as ColPali and ColNomic achieve strong accuracy by representing each document with hundreds to...
By Jianxin You, Kun Ni
arXiv:2606. 01790v1 Announce Type: cross Abstract: Vision-language-model-based graphical user interface (GUI) agents have shown broad automation capabilities, yet deployment is bottlenecked by a key-value (KV) cache that grows linearly with interaction steps.
By Yuhang Han, Wenzheng Yang, Yujie Chen, Xiangqi Jin, Yaojie Zhang, Siteng Huang, Linfeng Zhang
arXiv:2601.20107v3 Announce Type: replace-cross
Abstract: Recent Vision-Language Models (e.g., ColPali) enable fine-grained Visual Document Retrieval (VDR) but incur prohibitive multi-vector index st...
By Zhuchenyang Liu, Ziyu Hu, Yao Zhang, Yu Xiao
arXiv:2608. 12032v1 Announce Type: cross Abstract: Video diffusion transformers are costly to sample: every denoising step applies self-attention over a long 3D token sequence, a quadratic cost that dominates as resolution and duration grow.
By Enhuai Liu, Yunke Wang, Yutong Wang, Changming Sun, Chang Xu
We introduce Predictive State Retrieval (PSR), a task in which a model observes a short video prefix and a temporal question about an object's future state, then retrieves instances from other videos or images that depict that state. Unlike action anticipation, which predicts a label, moment retrieval, which localizes an observed event within a video, or video generation, which synthesizes pixels, PSR combines anticipation with cross-instance retrieval across multiple temporal horizons.