arXiv AI By Iishaan Inabathini, Margaret M. Henderson

Cross-attention encoding models reveal dynamic spatiotemporal routing across human higher visual cortex

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv Computer Vision
Oct 12

Neurons++: A Unified Visual-Stream Framework for Image and Video Reconstruction from fMRI with Neural-to-Visual Interpretability

Neurons++ is a visual‑stream framework that reconstructs both static images and dynamic videos from fMRI data. It builds on the earlier Neurons model by extending from video‑only to image‑and‑video reconstruction and replacing parallel task conditioning with a serial, cortex‑inspired decoding pipeline. The approach first pre‑trains on image‑fMRI pairs for stable spatial‑semantic initialization, then fine‑tunes separately for image and video tasks, organizing fMRI representations into spatial, semantic, language‑level, and temporal cues, and achieves state‑of‑the‑art video reconstruction with competitive image results while enabling neural‑to‑visual interpretability and ROI‑guided semantic modulation.

By Haonan Wang, Yuchen Wang, Jingyu Lu, Qixiang Zhang, Lehan Wang, Xuanqi Huang, Xiaomeng Li
arXiv Computer Vision
Oct 23

Learning from Next-Frame Prediction: Towards Unified and Scalable Visual Representation Learning

The paper introduces VANE, a self‑supervised visual pretraining framework that learns from next‑frame prediction in embedding space. It uses a causal encoder to summarize past observations, a state predictor to infer the representation of the entire next frame, and a lightweight generative decoder to reconstruct fine‑grained spatial detail. By applying the same objective to both videos and still images through a local‑to‑global crop sequence, VANE achieves state‑of‑the‑art performance on anticipation, recognition, and dense perception benchmarks, notably outperforming V‑JEPA on EPIC‑KITCHENS‑100 action anticipation.

By Yifeng Yao, Jinghan Li, Zhihao Zhang, Yajiao Xiong, Yang Jin, Kun Xu, Huishuai Zhang, Dongyan Zhao