arXiv Computer Vision

Predicting upcoming visual features during eye movements yields scene representations aligned with human visual cortex

arXiv Computer Vision
Oct 23

Learning from Next-Frame Prediction: Towards Unified and Scalable Visual Representation Learning

The paper introduces VANE, a self‑supervised visual pretraining framework that learns from next‑frame prediction in embedding space. It uses a causal encoder to summarize past observations, a state predictor to infer the representation of the entire next frame, and a lightweight generative decoder to reconstruct fine‑grained spatial detail. By applying the same objective to both videos and still images through a local‑to‑global crop sequence, VANE achieves state‑of‑the‑art performance on anticipation, recognition, and dense perception benchmarks, notably outperforming V‑JEPA on EPIC‑KITCHENS‑100 action anticipation.

By Yifeng Yao, Jinghan Li, Zhihao Zhang, Yajiao Xiong, Yang Jin, Kun Xu, Huishuai Zhang, Dongyan Zhao
arXiv Computer Vision
Nov 2

PPS: Plug-and-Play Saccadic Vision for Fine-Grained Classification

PPS: Plug-and-Play Saccadic Vision for Fine-Grained Classification proposes a lightweight extension that transforms any pretrained backbone with spatial feature maps into a saccadic classifier. The method adds only a small priority head and a single cross-attention fusion block, using a weight-tied encoder to process a downsampled peripheral glance and high-resolution glimpse crops selected by a priority distribution. A sequential sampler with non-maximum suppression draws a configurable number of glimpses, and a residual multi-head cross-attention block fuses the glance with the glimpses; the priority head is trained via a per-glimpse information-gain target from a lightweight memory bank, avoiding reinforcement learning. The approach achieves linear complexity in the number of glimpses and decouples peak memory from input resolution, consistently improving performance across nine fine-grained classification benchmarks.

By Johann Schmidt, Sebastian Stober, Joachim Denzler, Paul Bodesheim
Hugging Face Trending Papers
Jul 6

TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Driving

Bird's-Eye View (BEV) end-to-end instance prediction has emerged as a robust paradigm for autonomous driving perception, effectively mitigating the error propagation inherent in traditional modular pipelines. However, current state-of-the-art approaches rely predominantly on geometric supervision, such as occupancy regression and optical flow, effectively treating scene agents as generic moving obstacles.

arXiv Computer Vision
Oct 1

GazeFlow: From Human Gaze Behavior to Generative Egocentric Gaze Prediction

GazeFlow is a new framework for egocentric gaze prediction that models gaze as a joint distribution of temporal positions conditioned on both top‑down task cues and bottom‑up visual saliency. It employs conditional flow matching to iteratively transform Gaussian noise into realistic gaze trajectories, using a velocity field informed by video‑encoded visual features and global task queries. On standard benchmarks, GazeFlow outperforms existing methods on per‑frame metrics and produces trajectories that better reflect human gaze dynamics.

By Sheng Zhao, Weikai Lin, Yuhao Zhu
arXiv Computer Vision
Oct 5

Gaze Attention: Query-Adaptive Visual Routing for Efficient Multimodal LLMs

Gaze Attention is a new mechanism for multimodal large language models that selectively focuses on relevant visual regions during each generation step, rather than attending to all visual tokens. By grouping tokens into spatial regions and using learnable context tokens to retain global information, it reduces attention computation and visual key‑value entries by up to 90%. Experiments on 13 image and 6 video benchmarks show that Gaze Attention matches or outperforms dense‑attention baselines while using fewer visual resources.

By Junha Song, Byeongho Heo, Geonmo Gu, Jaegul Choo, Dongyoon Han, Sangdoo Yun
arXiv Computer Vision
Nov 2

ObjectGuidedMap: An Object-Aware Model for Robust Bird's Eye View Map Prediction in High-Occlusion Scenarios

ObjectGuidedMap introduces a multitask architecture that jointly predicts bird's eye view (BEV) maps and 3D objects, enabling explicit information exchange between the two tasks. By adding an object detection branch and cross‑task feature fusion through deformable attention and cross‑attention, the model allows map instances to leverage object context while remaining robust to occlusions. Experiments on nuScenes and Argoverse 2 show consistent improvements in BEV map prediction accuracy, especially a 21% mAP gain in heavily occluded scenes.

By Christian Bohn, Harsh Yadav, Klaus Friedrichs, Hasan Tercan, Tobias Meisen
arXiv Computer Vision
Oct 12

GuidedNav: Shaping Vision-Language Navigation Representations through Pre-action Attention and Spatial Guidance

GuidedNav is a new framework for vision‑and‑language navigation that explicitly shapes the representations linking instructions to actions. It uses subtask and landmark guidance to focus pre‑action attention on the relevant instruction segment and visual regions, and spatial guidance to align internal representations with geometric features of the environment. The method achieves state‑of‑the‑art success rates on R2R‑CE and RxR‑CE val‑unseen, improving by 8.1% and 13.1% respectively, while also providing faster inference and introducing two new annotation‑enriched datasets, Guided‑R2R and Guided‑RxR.

By Jingyu Guo, Jiaxin Huang, Chengxing Lin, Xiaoyu Luo, Cheng Wen, Shanshan Ye, Tingjin Chu, Shaoli Huang, Mingming Gong
arXiv Computer Vision
Oct 21

Gaze2HOI: Gaze-Aware Intention Disambiguation for Egocentric Hand-Object Interaction Forecasting

arXiv:2610.27877v1 Announce Type: new Abstract: Anticipating how users will interact with their surroundings is essential for proactive systems that provide timely assistance in AR, VR, and embodied...

By Chuanyang Ma, Chenye Wang, Yixuan Wang, Huiling Zhou, Xingqun Qi, Muyi Sun, Man Zhang, Caifeng Shan, Zhenan Sun