arXiv:2609.36366v1 Announce Type: cross
Abstract: Understanding how the brain parses actions and events from time-varying natural inputs is a central challenge in neuroscience. Recent work has used d...
By Iishaan Inabathini, Margaret M. Henderson
The paper introduces VANE, a self‑supervised visual pretraining framework that learns from next‑frame prediction in embedding space. It uses a causal encoder to summarize past observations, a state predictor to infer the representation of the entire next frame, and a lightweight generative decoder to reconstruct fine‑grained spatial detail. By applying the same objective to both videos and still images through a local‑to‑global crop sequence, VANE achieves state‑of‑the‑art performance on anticipation, recognition, and dense perception benchmarks, notably outperforming V‑JEPA on EPIC‑KITCHENS‑100 action anticipation.
By Yifeng Yao, Jinghan Li, Zhihao Zhang, Yajiao Xiong, Yang Jin, Kun Xu, Huishuai Zhang, Dongyan Zhao
PPS: Plug-and-Play Saccadic Vision for Fine-Grained Classification proposes a lightweight extension that transforms any pretrained backbone with spatial feature maps into a saccadic classifier. The method adds only a small priority head and a single cross-attention fusion block, using a weight-tied encoder to process a downsampled peripheral glance and high-resolution glimpse crops selected by a priority distribution. A sequential sampler with non-maximum suppression draws a configurable number of glimpses, and a residual multi-head cross-attention block fuses the glance with the glimpses; the priority head is trained via a per-glimpse information-gain target from a lightweight memory bank, avoiding reinforcement learning. The approach achieves linear complexity in the number of glimpses and decouples peak memory from input resolution, consistently improving performance across nine fine-grained classification benchmarks.
By Johann Schmidt, Sebastian Stober, Joachim Denzler, Paul Bodesheim
Bird's-Eye View (BEV) end-to-end instance prediction has emerged as a robust paradigm for autonomous driving perception, effectively mitigating the error propagation inherent in traditional modular pipelines. However, current state-of-the-art approaches rely predominantly on geometric supervision, such as occupancy regression and optical flow, effectively treating scene agents as generic moving obstacles.
GazeFlow is a new framework for egocentric gaze prediction that models gaze as a joint distribution of temporal positions conditioned on both top‑down task cues and bottom‑up visual saliency. It employs conditional flow matching to iteratively transform Gaussian noise into realistic gaze trajectories, using a velocity field informed by video‑encoded visual features and global task queries. On standard benchmarks, GazeFlow outperforms existing methods on per‑frame metrics and produces trajectories that better reflect human gaze dynamics.
By Sheng Zhao, Weikai Lin, Yuhao Zhu
arXiv:2610.30724v2 Announce Type: replace-cross
Abstract: Video provides a natural source of supervision: what happens next serves as a learning signal for visual world modeling. Yet masked video pre...
By Yifeng Yao, Jinghan Li, Zhihao Zhang, Yajiao Xiong, Yang Jin, Kun Xu, Huishuai Zhang, Dongyan Zhao
Gaze Attention is a new mechanism for multimodal large language models that selectively focuses on relevant visual regions during each generation step, rather than attending to all visual tokens. By grouping tokens into spatial regions and using learnable context tokens to retain global information, it reduces attention computation and visual key‑value entries by up to 90%. Experiments on 13 image and 6 video benchmarks show that Gaze Attention matches or outperforms dense‑attention baselines while using fewer visual resources.
By Junha Song, Byeongho Heo, Geonmo Gu, Jaegul Choo, Dongyoon Han, Sangdoo Yun
ObjectGuidedMap introduces a multitask architecture that jointly predicts bird's eye view (BEV) maps and 3D objects, enabling explicit information exchange between the two tasks. By adding an object detection branch and cross‑task feature fusion through deformable attention and cross‑attention, the model allows map instances to leverage object context while remaining robust to occlusions. Experiments on nuScenes and Argoverse 2 show consistent improvements in BEV map prediction accuracy, especially a 21% mAP gain in heavily occluded scenes.
By Christian Bohn, Harsh Yadav, Klaus Friedrichs, Hasan Tercan, Tobias Meisen
GuidedNav is a new framework for vision‑and‑language navigation that explicitly shapes the representations linking instructions to actions. It uses subtask and landmark guidance to focus pre‑action attention on the relevant instruction segment and visual regions, and spatial guidance to align internal representations with geometric features of the environment. The method achieves state‑of‑the‑art success rates on R2R‑CE and RxR‑CE val‑unseen, improving by 8.1% and 13.1% respectively, while also providing faster inference and introducing two new annotation‑enriched datasets, Guided‑R2R and Guided‑RxR.
By Jingyu Guo, Jiaxin Huang, Chengxing Lin, Xiaoyu Luo, Cheng Wen, Shanshan Ye, Tingjin Chu, Shaoli Huang, Mingming Gong
arXiv:2610.01942v1 Announce Type: new
Abstract: Predicting the future evolution of a scene is a fundamental capability for world modeling. Recent work has shown that operating in the feature space of...
By Efstathios Karypidis, Spyros Gidaris, Nikos Komodakis
arXiv:2609.36645v1 Announce Type: cross
Abstract: Future prediction is increasingly used to improve vision-language-action (VLA) policies, based on the premise that anticipating scene evolution encou...
By Hanseul Kim, Jewon Yeom, Youngjoon Jeong, Minsoo Jo, Taesup Kim
arXiv:2610.27877v1 Announce Type: new
Abstract: Anticipating how users will interact with their surroundings is essential for proactive systems that provide timely assistance in AR, VR, and embodied...
By Chuanyang Ma, Chenye Wang, Yixuan Wang, Huiling Zhou, Xingqun Qi, Muyi Sun, Man Zhang, Caifeng Shan, Zhenan Sun