arXiv AI
Jul 15
The TIME Machine: On The Power of Motion for Efficient Perception
arXiv:2605. 23045v2 Announce Type: replace-cross Abstract: Video representation learning has seen tremendous progress in recent years.
Image Classifiers are Efficient Self-Supervised Video Representation Learners
Related stories
arXiv AI
Aug 28
LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics
LeVJEPA is a video encoder that eliminates the need for architectural asymmetries, exponential-moving-average target encoders, stop-gradients, and capacity-limited predictors used in prior self‑supervised methods. It trains a single encoder with an invariance loss over global and local views, regularized by SIGReg to prevent collapse, and achieves strong performance with far less pretraining compute. The approach also allows block‑causal attention, making temporal ordering a property of the encoder itself, and matches or surpasses state‑of‑the‑art baselines on both appearance‑centric and motion‑centric benchmarks.
arXiv AI
Jun 16
You Don't Need Strong Assumptions: Visual Representation Learning via Temporal Differences
arXiv:2606. 15956v1 Announce Type: cross Abstract: Progress in AI has largely been driven by methods that assume less.
arXiv Computer Vision
Sep 7
Object Concepts Emerge from Motion
The paper introduces a biologically inspired framework that learns object‑centric visual representations from raw videos without human annotations or camera calibration. By using motion boundaries detected via optical flow and clustering to create pseudo‑instance masks, the method supervises a single‑image encoder with pixel‑level pairwise metric learning. Training on 195 million pseudo‑labeled frames and expanding to 421 million frames through Motion‑Verified Self‑Training, the approach yields Swin‑based encoders that outperform or match supervised and self‑supervised baselines on tasks such as monocular depth estimation, 3D object detection, 3D occupancy prediction, and end‑to‑end planning.
Hugging Face Trending Papers
Aug 13
V-RAE: Rethinking Video Latent Spaces for Generation
Latent video generation relies on autoencoders to define a compact space in which generative models operate. Although video autoencoder architectures have evolved substantially, their latent spaces are still optimized primarily for pixel-level reconstruction and provide limited high-level semantic organization.
arXiv AI
Jun 16
Momentum-Guided Semantic Forecasting (MoFore) for Self-Supervised Video Representation Learning
arXiv:2606. 14765v1 Announce Type: cross Abstract: Self-supervised video representation learning has recently advanced through contrastive learning, masked reconstruction, and predictive representation learning.
arXiv Computer Vision
2d ago
TT-VidT: Decoupling the Temporal Axis for Efficient Motion-Centric Video Pretraining
TT-VidT is a video pretraining method that decouples the temporal axis by combining a per‑frame ViT-B/16 spatial encoder with a compact Temporal Transfer Layer trained via Diff Compression. The authors conduct a systematic 24‑configuration study to isolate architecture, objective, and decoder effects, showing that the full TT-VidT design yields the strongest motion‑sensitive representations. In downstream fine‑tuning, TT‑VidT outperforms state‑of‑the‑art baselines on Jester, Something‑Something V2, ARID, and Diving48 while using significantly fewer encoder FLOPs.
arXiv AI
Jul 14
On Occlusions in Video Action Detection: Benchmark Datasets And Training Recipes
arXiv:2410. 19553v2 Announce Type: replace-cross Abstract: This paper explores the impact of occlusions in video action detection.
arXiv Machine Learning
Aug 4
Asleep at the Wheel: JEPA's Limitations in Evaluating Novel Driving Data
arXiv:2608. 01336v1 Announce Type: cross Abstract: Modern autonomous-driving fleets record far more video than human reviewers can inspect.
arXiv AI
Sep 1
PRISM: Predictive Recomposition via Semantic Latent Decomposition for View-invariant Video Representation Learning
arXiv:2608.30388v1 Announce Type: cross Abstract: Cross-view video representation learning aims to capture viewpoint-invariant action semantics despite substantial appearance changes across egocentri...
arXiv Computer Vision
4d ago
A Controlled Study of Self-Supervised Image and Video Pretraining under Limited Resources
The paper reports a controlled study of self‑supervised learning (SSL) objectives for image and video pretraining under limited data, architecture, and compute budgets. It compares contrastive, reconstruction, feature‑prediction, and diffusion methods, finding that DINOv2‑style pretraining delivers the best overall performance. Combining DINOv2 with video SSL objectives such as VideoMAE improves image classification and segmentation but harms video tracking and camera‑pose estimation, highlighting a trade‑off between semantic and geometric learning.
arXiv Computer Vision
Sep 7
Intrinsic Temporal Adaptation of CLIP for Partially Relevant Video Retrieval
The paper introduces Intrinsic Temporal Adaptation (ITA) for Partially Relevant Video Retrieval (PRVR), a task that seeks untrimmed videos containing moments relevant to a text query. ITA employs a Backbone-Internal Temporal Adaptation that lets the final visual transformer layers attend to neighboring frames, creating temporally aware embeddings while keeping CLIP frozen. Additionally, an Affinity-Weighted Gradient Propagation technique softly aggregates top‑k frames based on text‑frame affinities to better handle the weakly supervised nature of PRVR, leading to state‑of‑the‑art performance and more accurate frame‑level evidence retrieval.