TrajTok introduces an end‑to‑end video tokenizer that replaces traditional patchification with a lightweight, trajectory‑based tokenization process. It performs implicit clustering over space‑time pixels to generate object trajectories in a single forward pass, dynamically adjusting token granularity to semantic complexity while remaining independent of video duration. The resulting TrajViT2 model outperforms existing methods on classification and retrieval benchmarks, and TrajTok can also serve as a probing head (TrajAdapter) or alignment connector (TrajVLM) for vision‑language tasks, notably improving long‑video reasoning.
arXiv:2608. 03112v1 Announce Type: cross Abstract: Vision-language models excel at image and video understanding but suffer from high inference latency due to the need to process thousands of tokens per image, limiting their deployment on resource-constrained edge devices and in real-time surveillance applications.
arXiv:2610.00686v1 Announce Type: new Abstract: Recent video-based world models pair the scalability of autoregressive (AR) prediction with the visual quality of diffusion models. The choice of scene...
arXiv:2609.39924v1 Announce Type: cross Abstract: Vision-language models face a fundamental scaling bottleneck: the number of visual tokens grows with both temporal duration and spatial resolution, m...
arXiv:2604.03414v2 Announce Type: replace Abstract: Video Large Language Models (Video LLMs) achieve strong performance on video understanding tasks but suffer from high inference costs due to the la...
arXiv:2606.06158v2 Announce Type: replace Abstract: Adaptive video tokenisation seeks to dynamically allocate token budgets based on the underlying visual complexity of a sequence. Current continuous...
Dual-State Slot Attention (DSSA) is a self‑supervised framework that separates per‑frame appearance from temporally stable identity within each slot, addressing conflicts that cause slot swapping in video object‑centric learning. It introduces a learned recurrent transition for identity updates and competition‑modulated aggregation to suppress weakly matching slots, thereby improving segmentation quality and temporal consistency. Experiments on MOVi‑C, MOVi‑D, and YouTube‑VIS show DSSA outperforms prior methods and enhances downstream object recognition and video dynamics prediction.
Video large language models (VLLMs) need long visual token sequences, making token compression essential for efficient inference. This paper introduces a training‑free token compression technique that leverages existing video codec information—such as reference‑frame dependencies, motion vectors, and residuals—to guide low‑cost search priors. By aligning these codec cues with the visual token grid, the method creates compact candidate pools for token protection and localized candidates for temporal token merging, reducing feature comparisons while maintaining performance comparable to or better than the FlashVID baseline across multiple VLLMs and benchmarks.
VideoTIR introduces a reinforcement‑learning approach to improve long‑video understanding by encouraging multimodal large language models to use comprehensive multi‑level toolkits efficiently. It combines Zero‑RL and SFT cold‑starting strategies to help models retrieve and focus on meaningful video segments, images, and regions, thereby reducing hallucinations. The method includes Toolkit Action Grouped Policy Optimization (TAGPO) to streamline tool‑calling and a sandbox‑based trajectory synthesis framework for high‑quality data, achieving strong results on three long‑video QA benchmarks.
Video large language models (VLLMs) need long visual token sequences, making token compression essential for efficient inference. This paper introduces a training‑free token compression method that leverages existing video codec information—such as reference‑frame dependencies, motion vectors, and residuals—to guide low‑cost search for token protection and temporal merging. Experiments on three VLLMs and four benchmarks show that the method matches or surpasses the FlashVID baseline while dramatically reducing feature comparisons.
In video understanding, vision-language models (VLMs) must ingest massive numbers of visual tokens, causing the computational and memory cost of the prefill stage to rise sharply. Such visual sequences are highly redundant along the spatio-temporal dimension, yet a high compression ratio is often accompanied by the loss of critical details.
arXiv:2608.20473v1 Announce Type: new Abstract: Video language models process videos as dense visual-token sequences with substantial representational redundancy. Compressing these sequences is there...
arXiv:2609.16722v1 Announce Type: new Abstract: Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates co...