arXiv Computer Vision By Matteo Dunnhofer, Christian Micheloni, Kohitij Kar

Primate vision reveals a missing principle for robust dynamic AI

Read the original on arXiv Computer Vision →

The study investigates how intelligent visual systems integrate object appearance with motion while maintaining robustness to appearance changes. By comparing human perception, macaque inferior temporal cortex activity, and various neural network models, the authors find that temporal integration enhances object representations, yet most video models fail to generalize when appearance varies. Predictive world models show the best cross‑appearance generalization and neural fidelity, though none fully replicate the cortical shift from appearance‑dominated to motion‑invariant coding.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI
Oct 1

Future Video Generation Better Aligns with the Human Visual Cortex than Observed Video

The study investigates how internal representations of video diffusion models align with human visual cortex responses. It finds that representations used for future video generation in an autoregressive (AR) model better match cortical activity than those for observed video, with future‑generation alignment concentrated in higher‑order visual areas. A behavioral experiment further shows that humans prefer videos enhanced by layers that align more strongly with cortical responses.

By Chang-Bae Bang, Hyungjin Chung, Byung-Hoon Kim
arXiv Computer Vision
Nov 2

Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling

The paper introduces GenSight, a unified embodied world model that combines future video generation and dense perception into a single pretrained video diffusion transformer. GenSight employs a shared spatiotemporal representation for video, depth, embodiment segmentation, and first‑frame‑relative 2D flow, and uses lightweight task modulation to predict all modalities within the same architecture. The model supports both multi‑step future video generation with aligned dense predictions and single‑step dense perception from an observed video, demonstrating strong performance on embodied manipulation data across multiple robot embodiments and human‑hand interactions.

By Haitang Feng, Zhuoman Liu, Zhennan Wang, Sheng Wang, Juelin Li, Chengzhi Zhao, Hexiang Wei, Lin Shao