Predicting upcoming visual features during eye movements yields scene representations aligned with human visual cortex
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The Flow has not summarised this story yet — read it at arXiv Computer Vision.
arXiv:2609.36366v1 Announce Type: cross Abstract: Understanding how the brain parses actions and events from time-varying natural inputs is a central challenge in neuroscience. Recent work has used d...
The paper introduces VANE, a self‑supervised visual pretraining framework that learns from next‑frame prediction in embedding space. It uses a causal encoder to summarize past observations, a state predictor to infer the representation of the entire next frame, and a lightweight generative decoder to reconstruct fine‑grained spatial detail. By applying the same objective to both videos and still images through a local‑to‑global crop sequence, VANE achieves state‑of‑the‑art performance on anticipation, recognition, and dense perception benchmarks, notably outperforming V‑JEPA on EPIC‑KITCHENS‑100 action anticipation.
PPS: Plug-and-Play Saccadic Vision for Fine-Grained Classification proposes a lightweight extension that transforms any pretrained backbone with spatial feature maps into a saccadic classifier. The method adds only a small priority head and a single cross-attention fusion block, using a weight-tied encoder to process a downsampled peripheral glance and high-resolution glimpse crops selected by a priority distribution. A sequential sampler with non-maximum suppression draws a configurable number of glimpses, and a residual multi-head cross-attention block fuses the glance with the glimpses; the priority head is trained via a per-glimpse information-gain target from a lightweight memory bank, avoiding reinforcement learning. The approach achieves linear complexity in the number of glimpses and decouples peak memory from input resolution, consistently improving performance across nine fine-grained classification benchmarks.
Bird's-Eye View (BEV) end-to-end instance prediction has emerged as a robust paradigm for autonomous driving perception, effectively mitigating the error propagation inherent in traditional modular pipelines. However, current state-of-the-art approaches rely predominantly on geometric supervision, such as occupancy regression and optical flow, effectively treating scene agents as generic moving obstacles.
GazeFlow is a new framework for egocentric gaze prediction that models gaze as a joint distribution of temporal positions conditioned on both top‑down task cues and bottom‑up visual saliency. It employs conditional flow matching to iteratively transform Gaussian noise into realistic gaze trajectories, using a velocity field informed by video‑encoded visual features and global task queries. On standard benchmarks, GazeFlow outperforms existing methods on per‑frame metrics and produces trajectories that better reflect human gaze dynamics.
arXiv:2610.30724v2 Announce Type: replace-cross Abstract: Video provides a natural source of supervision: what happens next serves as a learning signal for visual world modeling. Yet masked video pre...