The study investigates how long‑video language models decide which frames to keep, compress, and reuse, testing each decision in isolation across six selection rules, three benchmarks, and two answering models. It finds that selecting frames based on queries yields the biggest performance boost, that halving spatial resolution costs little, and that reallocating saved tokens to more compressed frames can further improve accuracy. The work also highlights the importance of a unified evaluation harness to avoid misleading comparisons.
arXiv:2607.28571v2 Announce Type: replace Abstract: Operational Earth observation increasingly calls for answering queries such as ``find the image pairs where a new building appeared.'' This means s...
VETO (Video Efficient Token Optimization for Vision Language Models) is a plug‑in that reduces the quadratic cost of visual tokens in long‑video inference by applying dual‑axis compression: an intra‑frame compressor merges semantically similar tokens within each frame, and an inter‑frame compressor merges temporally redundant frames. By first compressing spatial dimensions, VETO lowers the cost of subsequent global temporal matching, surpassing single‑axis methods and achieving up to 45% faster inference on models such as LLaVA‑OneVision‑7B while maintaining or improving accuracy. The approach is universally applicable across LLaVA‑OneVision, InternVL‑2.5, and LongVA, preserving or enhancing zero‑shot accuracy even under extreme token budgets.
arXiv:2609.39096v1 Announce Type: new Abstract: Autoregressive video diffusion supports streaming generation and interactive control, but its KV cache grows with the generated history. Existing compr...
arXiv:2608.22526v1 Announce Type: new Abstract: We introduce RS$^3$-Prune, a training-free token-pruning recipe that instantiates as a small set of inference time hooks atop existing video object seg...
arXiv:2608.05707v2 Announce Type: replace Abstract: Frame selection is essential for applying Large Multimodal Models (LMMs) to long videos due to severe frame redundancy and limited context windows....
arXiv:2609.09528v1 Announce Type: new Abstract: Video large language models (Video-LLMs) are increasingly used as the perceptual front end of world models, a role that assumes they can read motion: h...
arXiv:2608. 03918v1 Announce Type: cross Abstract: Efficient long-video understanding requires vision--language models (VLMs) to reason over a small number of frames selected as sparse visual evidence.
The paper evaluates coreset selection methods by incorporating both selection and training time into a unified wall‑clock budget, using a standardized benchmark across four datasets and multiple selectors. Across numerous budget anchors, simple random or full‑data training consistently outperforms sophisticated selectors, and selection costs are dominated by a full‑dataset scan that cannot be amortized. The study also identifies when subset reuse can justify selection and reports several correctness fixes in a popular codebase.
arXiv:2607. 22771v1 Announce Type: cross Abstract: Picking the frozen image encoder for a 3D~CT vision--language model (VLM), together with the token-compression scheme on top of it, is a search over many candidates.
arXiv:2608. 12032v1 Announce Type: cross Abstract: Video diffusion transformers are costly to sample: every denoising step applies self-attention over a long 3D token sequence, a quadratic cost that dominates as resolution and duration grow.
The paper introduces FrameBlend, a plug‑and‑play technique for streaming video understanding that selectively recomputes key‑value (KV) caches at query time. By re‑prefilling only query‑relevant video tokens, FrameBlend allows shorter attention windows during idle prefill, cutting prefill cost while boosting accuracy. Experiments on three SVU frameworks and five benchmarks show up to 3.9‑point accuracy gains and significant reductions in FLOPs and processing time.
The paper introduces Rate‑Controlled Refresh (RCR), a method for re‑selecting key‑value (KV) cache entries during long‑form multimodal generation. RCR lets attention drift determine when and where to refresh, while a per‑sample controller limits how often this occurs, and a low‑rank drift detector keeps the process GPU‑resident. Across multiple models and tasks, RCR outperforms the per‑step retrieval method InfiniGen by up to 3.8 ROUGE‑L points and remains close to exact per‑step re‑selection, with reduced data movement and a smaller GPU‑resident detector.