arXiv Computer Vision

Primate vision reveals a missing principle for robust dynamic AI

The study investigates how intelligent visual systems integrate object appearance with motion while maintaining robustness to appearance changes. By comparing human perception, macaque inferior temporal cortex activity, and various neural network models, the authors find that temporal integration enhances object representations, yet most video models fail to generalize when appearance varies. Predictive world models show the best cross‑appearance generalization and neural fidelity, though none fully replicate the cortical shift from appearance‑dominated to motion‑invariant coding.

arXiv AI
Oct 1

Future Video Generation Better Aligns with the Human Visual Cortex than Observed Video

The study investigates how internal representations of video diffusion models align with human visual cortex responses. It finds that representations used for future video generation in an autoregressive (AR) model better match cortical activity than those for observed video, with future‑generation alignment concentrated in higher‑order visual areas. A behavioral experiment further shows that humans prefer videos enhanced by layers that align more strongly with cortical responses.

By Chang-Bae Bang, Hyungjin Chung, Byung-Hoon Kim
arXiv Computer Vision
Nov 2

Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling

The paper introduces GenSight, a unified embodied world model that combines future video generation and dense perception into a single pretrained video diffusion transformer. GenSight employs a shared spatiotemporal representation for video, depth, embodiment segmentation, and first‑frame‑relative 2D flow, and uses lightweight task modulation to predict all modalities within the same architecture. The model supports both multi‑step future video generation with aligned dense predictions and single‑step dense perception from an observed video, demonstrating strong performance on embodied manipulation data across multiple robot embodiments and human‑hand interactions.

By Haitang Feng, Zhuoman Liu, Zhennan Wang, Sheng Wang, Juelin Li, Chengzhi Zhao, Hexiang Wei, Lin Shao
arXiv Computer Vision
Oct 12

Neurons++: A Unified Visual-Stream Framework for Image and Video Reconstruction from fMRI with Neural-to-Visual Interpretability

Neurons++ is a visual‑stream framework that reconstructs both static images and dynamic videos from fMRI data. It builds on the earlier Neurons model by extending from video‑only to image‑and‑video reconstruction and replacing parallel task conditioning with a serial, cortex‑inspired decoding pipeline. The approach first pre‑trains on image‑fMRI pairs for stable spatial‑semantic initialization, then fine‑tunes separately for image and video tasks, organizing fMRI representations into spatial, semantic, language‑level, and temporal cues, and achieves state‑of‑the‑art video reconstruction with competitive image results while enabling neural‑to‑visual interpretability and ROI‑guided semantic modulation.

By Haonan Wang, Yuchen Wang, Jingyu Lu, Qixiang Zhang, Lehan Wang, Xuanqi Huang, Xiaomeng Li
arXiv Computer Vision
6d ago

A probabilistic model of visual segmentation explains early visual cortical dynamics

The paper presents a biologically plausible model of visual cortex computation that couples feature inference with segmentation inference, generating neural dynamics through iterative recurrence and feedback. The model predicts gain and variability changes in evoked responses to natural images and reproduces classic single‑neuron findings such as gain dynamics and variability decay during stimulus presentation. It further suggests that these dynamics are organized around inferred segment structure and are more heterogeneous than previously thought, offering a normative framework that could inform both experimental and artificial network designs.

By Tridib K. Biswas, Ruben Coen-Cagli
Hugging Face Trending Papers
Jun 3

Coarse-to-fine Hierarchical Architecture with Sequential Mamba for Brain Reconstruction

Understanding the relationship between deep visual representations and the human visual system is a fundamental challenge in computational neuroscience. While modern vision models achieve strong performance in image recognition, their correspondence with the hierarchical organization of the human visual cortex remains an open question.

arXiv Computer Vision
Oct 19

Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning

Dual-State Slot Attention (DSSA) is a self‑supervised framework that separates per‑frame appearance from temporally stable identity within each slot, addressing conflicts that cause slot swapping in video object‑centric learning. It introduces a learned recurrent transition for identity updates and competition‑modulated aggregation to suppress weakly matching slots, thereby improving segmentation quality and temporal consistency. Experiments on MOVi‑C, MOVi‑D, and YouTube‑VIS show DSSA outperforms prior methods and enhances downstream object recognition and video dynamics prediction.

By Sieu Tran, Duc Nguyen, Hao Vo, Khoa Vo, Ngan Le