arXiv Machine Learning By Amir Hosein Fadaei, Mahyar Maleki, Mohammad-Reza A. Dehaqani

Brain-Aligned Multi-Stream Video Transformers with Sparse Self-Selection

Read the original on arXiv Machine Learning →

arXiv:2607. 17625v1 Announce Type: cross Abstract: Modern video transformers typically ignore principles from primate vision and are rarely evaluated against neural data, limiting their biological interpretability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Aug 28

LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics

LeVJEPA is a video encoder that eliminates the need for architectural asymmetries, exponential-moving-average target encoders, stop-gradients, and capacity-limited predictors used in prior self‑supervised methods. It trains a single encoder with an invariance loss over global and local views, regularized by SIGReg to prevent collapse, and achieves strong performance with far less pretraining compute. The approach also allows block‑causal attention, making temporal ordering a property of the encoder itself, and matches or surpasses state‑of‑the‑art baselines on both appearance‑centric and motion‑centric benchmarks.

By Lukas Kuhn, Lucas Maes, Giuseppe Serra, Quentin Le Lidec, Yann LeCun, Randall Balestriero, Florian Buettner
arXiv Computer Vision
3d ago

Sparse Attention for Video Generation Acceleration via Growing Sparsity and Reduced Search

The paper introduces Growing Sparsity and Reduced Search (GSRS), a training‑free framework that accelerates Diffusion Transformers for video generation by progressively increasing sparsity and reusing early attention blocks to limit search space. GSRS uses Top‑p selection for adaptive sparsity and a precise evaluation scheme that computes exact attention weights with negligible overhead. Experiments on HunyuanVideo and Wan 2.1 show 1.55×–2.21× speedups while preserving generation quality comparable to full attention.

By Yun Dai, Yuyun Chen, Jiarui Wen, Huiping Zhuang, Cen Chen, Ziqian Zeng
arXiv Computer Vision
Oct 12

Neurons++: A Unified Visual-Stream Framework for Image and Video Reconstruction from fMRI with Neural-to-Visual Interpretability

Neurons++ is a visual‑stream framework that reconstructs both static images and dynamic videos from fMRI data. It builds on the earlier Neurons model by extending from video‑only to image‑and‑video reconstruction and replacing parallel task conditioning with a serial, cortex‑inspired decoding pipeline. The approach first pre‑trains on image‑fMRI pairs for stable spatial‑semantic initialization, then fine‑tunes separately for image and video tasks, organizing fMRI representations into spatial, semantic, language‑level, and temporal cues, and achieves state‑of‑the‑art video reconstruction with competitive image results while enabling neural‑to‑visual interpretability and ROI‑guided semantic modulation.

By Haonan Wang, Yuchen Wang, Jingyu Lu, Qixiang Zhang, Lehan Wang, Xuanqi Huang, Xiaomeng Li