arXiv Machine Learning

Receptive-field-constrained stimulus optimization for human early and intermediate visual cortex

arXiv AI
3d ago

Prototypes of the Mind: A Unified Framework for Probing the Visual Brain

The paper presents a unified framework that links vision foundation models to fMRI brain activity using sparse, interpretable visual prototypes. By replacing black‑box features with these prototypes and applying a linear readout, the model reveals which visual concepts drive neural responses, maps cortical organization through voxel tuning, and enables controlled stimulus synthesis to probe cortical selectivity. This approach provides a transparent, prototype‑based representation for studying visual representations in the human brain.

By Shi Chen, Seoyoung Ahn, Doris Tsao
arXiv Computer Vision
Oct 5

FiberGeoText: A Vision-Language Model for Population- Level Organization of Superficial White Matter

FiberGeoText (FGT) is a vision‑language model that clusters short‑range superficial white matter streamlines from ultra‑high‑resolution diffusion MRI into population‑level groups. It jointly encodes each streamline’s 3‑D trajectory, cortical anatomical context (via text from multiple parcellation schemes), and shape, using a pretrained large language model to unify heterogeneous anatomical descriptions. Evaluations on 0.76 mm diffusion data show that FGT outperforms state‑of‑the‑art methods in cortical parcel coherence, shape consistency, cluster‑size consistency, and cross‑subject correspondence, and it generalizes well to unseen subjects, recovering 96.7 % of learned clusters.

By Yuqian Chen, R. Jarrett Rushmore, Guikun Chen, Fan Zhang, Edward Yeterian, Nikos Makris, Yogesh Rathi, Lauren J. O'Donnell
Hugging Face Trending Papers
Jun 3

Coarse-to-fine Hierarchical Architecture with Sequential Mamba for Brain Reconstruction

Understanding the relationship between deep visual representations and the human visual system is a fundamental challenge in computational neuroscience. While modern vision models achieve strong performance in image recognition, their correspondence with the hierarchical organization of the human visual cortex remains an open question.

arXiv Computer Vision
Oct 16

FlatClip: Reusing Image Foundation Models for fMRI Representation Learning via Cortical Flatmaps

FlatClip is a method that leverages frozen image foundation models to predict fMRI activity by mapping cortical signals onto anatomy‑linked flatmaps and extracting features with a SigLIP2 encoder. Only a lightweight downstream probe is trained, avoiding fMRI‑specific encoder pretraining or fine‑tuning. Experiments on four resting‑state benchmarks show FlatClip surpasses ROI‑level baselines on HCP and ADNI, while its performance varies on PPMI and remains below the best voxel‑level models; on NSD visual‑fMRI decoding, restricting inputs to task‑active cortex improves recognition, and spatial perturbations confirm the importance of cortical arrangement and input‑region selection.

By Mo Wang, Wenhao Ye, Zihan Ning, Jiayu Zuo, Junfeng Xia, Hongkai Wen, Quanying Liu
arXiv Computer Vision
Aug 28

NEvo: Neural-Guided Evolutionary Video Synthesis for Dynamic Visual Selectivity

NEvo is a neural‑guided evolutionary video synthesis framework that generates dynamic stimuli optimized for specific brain regions in the visual cortex. It performs evolutionary search over a structured prompt space, guided by a dynamic encoding model that predicts voxel‑level responses to video inputs, thereby discovering hyper‑activating videos that outperform handcrafted localizers. The synthesized videos recover known selectivities across ventral, dorsal, and lateral pathways and reveal systematic differences in sensitivity to temporal dynamics, offering new insights into the progression of social‑dynamic features along the lateral stream.

By Yingtian Tang, Sogand Salehi, Ming Zhou, Amir Zamir, Leyla Isik, Martin Schrimpf
arXiv Computer Vision
Nov 9

Structured Visual Landscape: Generating Preferred Representations in Biological and Artificial Neural Networks

The paper introduces Vis‑Lens, a framework that combines a variational autoencoder and a diffusion model to generate natural, realistic images that match preferred visual representations in both artificial neural networks and the human visual cortex. By applying an activation‑based prior, the method regularizes a structured visual representation landscape, enabling direct modification of latent codes without per‑image optimization. Experiments using fMRI data demonstrate that Vis‑Lens improves activation modulation and image realism compared to recent state‑of‑the‑art approaches.

By Yusong Zhao, Edgar Y. Walker, Hao Wang, Mi Lu
arXiv Machine Learning
Sep 24

The Computational Value of Sensory-Aligned Receptive Fields Depends on Neuronal Expressivity

The study investigates whether sensory-aligned receptive fields provide computational benefits beyond mere resource efficiency in recurrent networks of Expressive Leaky Memory neurons. Across auditory and event-based visual classification tasks, receptive fields aligned with task-relevant sensory coordinates improve test accuracy compared to budget-matched random fields, but this advantage disappears when coordinates are scrambled or irrelevant. The benefit diminishes as neuronal expressivity increases, and generic synaptic sparsity regularization only partially recovers performance, indicating that structured receptive fields act as a computational prior beyond sparsity alone.

By Agnese Adorante, Aaron Spieler, Anna Levina