arXiv Computer Vision By Yukai Huang, Haoyu Albert Wang, Jiawei Zhang, Jialun Ma, Jiayao Ding, Yiran Lin, Xingyu Chen, Ziqi Gao, Yuguo Yu

Decoding spatial and semantic information from large-scale visual cortical activity for natural-movie reconstruction

Read the original on arXiv Computer Vision →

The paper introduces Ca2REAL, a neural-to-video framework that uses calcium imaging data from 7,863 V1 neurons to reconstruct natural movie clips. By conditioning a pretrained video diffusion model with layout and semantic controls derived from neural activity, Ca2REAL achieves 81.4% stimulus-identification accuracy and outperforms deterministic baselines in perceptual and semantic metrics. Additional tests on a virtual population of 116,705 neurons confirm that reconstruction relies on distributed, stimulus-specific population structure, with layout and semantic pathways providing complementary spatial and identity information.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision
Oct 12

Neurons++: A Unified Visual-Stream Framework for Image and Video Reconstruction from fMRI with Neural-to-Visual Interpretability

Neurons++ is a visual‑stream framework that reconstructs both static images and dynamic videos from fMRI data. It builds on the earlier Neurons model by extending from video‑only to image‑and‑video reconstruction and replacing parallel task conditioning with a serial, cortex‑inspired decoding pipeline. The approach first pre‑trains on image‑fMRI pairs for stable spatial‑semantic initialization, then fine‑tunes separately for image and video tasks, organizing fMRI representations into spatial, semantic, language‑level, and temporal cues, and achieves state‑of‑the‑art video reconstruction with competitive image results while enabling neural‑to‑visual interpretability and ROI‑guided semantic modulation.

By Haonan Wang, Yuchen Wang, Jingyu Lu, Qixiang Zhang, Lehan Wang, Xuanqi Huang, Xiaomeng Li
Hugging Face Trending Papers
Jun 3

Coarse-to-fine Hierarchical Architecture with Sequential Mamba for Brain Reconstruction

Understanding the relationship between deep visual representations and the human visual system is a fundamental challenge in computational neuroscience. While modern vision models achieve strong performance in image recognition, their correspondence with the hierarchical organization of the human visual cortex remains an open question.