The TIME Machine: On The Power of Motion for Efficient Perception
arXiv:2605. 23045v2 Announce Type: replace-cross Abstract: Video representation learning has seen tremendous progress in recent years.
arXiv:2605. 23045v2 Announce Type: replace-cross Abstract: Video representation learning has seen tremendous progress in recent years.
LeVJEPA is a video encoder that eliminates the need for architectural asymmetries, exponential-moving-average target encoders, stop-gradients, and capacity-limited predictors used in prior self‑supervised methods. It trains a single encoder with an invariance loss over global and local views, regularized by SIGReg to prevent collapse, and achieves strong performance with far less pretraining compute. The approach also allows block‑causal attention, making temporal ordering a property of the encoder itself, and matches or surpasses state‑of‑the‑art baselines on both appearance‑centric and motion‑centric benchmarks.
arXiv:2606. 15956v1 Announce Type: cross Abstract: Progress in AI has largely been driven by methods that assume less.
The paper introduces a biologically inspired framework that learns object‑centric visual representations from raw videos without human annotations or camera calibration. By using motion boundaries detected via optical flow and clustering to create pseudo‑instance masks, the method supervises a single‑image encoder with pixel‑level pairwise metric learning. Training on 195 million pseudo‑labeled frames and expanding to 421 million frames through Motion‑Verified Self‑Training, the approach yields Swin‑based encoders that outperform or match supervised and self‑supervised baselines on tasks such as monocular depth estimation, 3D object detection, 3D occupancy prediction, and end‑to‑end planning.
Dual-State Slot Attention (DSSA) is a self‑supervised framework that separates per‑frame appearance from temporally stable identity within each slot, addressing conflicts that cause slot swapping in video object‑centric learning. It introduces a learned recurrent transition for identity updates and competition‑modulated aggregation to suppress weakly matching slots, thereby improving segmentation quality and temporal consistency. Experiments on MOVi‑C, MOVi‑D, and YouTube‑VIS show DSSA outperforms prior methods and enhances downstream object recognition and video dynamics prediction.
arXiv:2610.30724v2 Announce Type: replace-cross Abstract: Video provides a natural source of supervision: what happens next serves as a learning signal for visual world modeling. Yet masked video pre...
Latent video generation relies on autoencoders to define a compact space in which generative models operate. Although video autoencoder architectures have evolved substantially, their latent spaces are still optimized primarily for pixel-level reconstruction and provide limited high-level semantic organization.
The paper introduces VANE, a self‑supervised visual pretraining framework that learns from next‑frame prediction in embedding space. It uses a causal encoder to summarize past observations, a state predictor to infer the representation of the entire next frame, and a lightweight generative decoder to reconstruct fine‑grained spatial detail. By applying the same objective to both videos and still images through a local‑to‑global crop sequence, VANE achieves state‑of‑the‑art performance on anticipation, recognition, and dense perception benchmarks, notably outperforming V‑JEPA on EPIC‑KITCHENS‑100 action anticipation.
arXiv:2606. 14765v1 Announce Type: cross Abstract: Self-supervised video representation learning has recently advanced through contrastive learning, masked reconstruction, and predictive representation learning.
TT-VidT is a video pretraining method that decouples the temporal axis by combining a per‑frame ViT-B/16 spatial encoder with a compact Temporal Transfer Layer trained via Diff Compression. The authors conduct a systematic 24‑configuration study to isolate architecture, objective, and decoder effects, showing that the full TT-VidT design yields the strongest motion‑sensitive representations. In downstream fine‑tuning, TT‑VidT outperforms state‑of‑the‑art baselines on Jester, Something‑Something V2, ARID, and Diving48 while using significantly fewer encoder FLOPs.
arXiv:2610.33857v1 Announce Type: new Abstract: We introduce TrackTok, an efficient video tokenization framework that produces object-centric tokens, i.e., tokens that consistently encode the same se...
The paper introduces DMSNet, a deformable adaptive multi‑scale edge‑guided attention network for video capsule endoscopy (VCE) classification. It evaluates DMSNet and 18 other architectures on the Kvasir‑Capsule dataset using both random frame‑level and source‑video‑disjoint validation protocols, reporting MCC scores of 0.442 and 0.456 respectively. The study also examines cross‑dataset performance on CV2024, the impact of source‑video diversity, and shows that model rankings and accuracy vary significantly with validation protocol, training source diversity, and external dataset shift.