FlatClip is a method that leverages frozen image foundation models to predict fMRI activity by mapping cortical signals onto anatomy‑linked flatmaps and extracting features with a SigLIP2 encoder. Only a lightweight downstream probe is trained, avoiding fMRI‑specific encoder pretraining or fine‑tuning. Experiments on four resting‑state benchmarks show FlatClip surpasses ROI‑level baselines on HCP and ADNI, while its performance varies on PPMI and remains below the best voxel‑level models; on NSD visual‑fMRI decoding, restricting inputs to task‑active cortex improves recognition, and spatial perturbations confirm the importance of cortical arrangement and input‑region selection.
arXiv:2609.31204v1 Announce Type: cross Abstract: Recent fMRI foundation models differ substantially in the spatial scale at which they represent brain activity. ROI- and connectivity-based models ar...
arXiv:2606. 04772v1 Announce Type: cross Abstract: Understanding the relationship between deep visual representations and the human visual system is a fundamental challenge in computational neuroscience.
Understanding the relationship between deep visual representations and the human visual system is a fundamental challenge in computational neuroscience. While modern vision models achieve strong performance in image recognition, their correspondence with the hierarchical organization of the human visual cortex remains an open question.
arXiv:2609.34167v2 Announce Type: replace Abstract: Foundation models pre-trained on large-scale fMRI datasets have shown strong downstream performance, but at substantial data and computation cost....
arXiv:2606. 11500v1 Announce Type: cross Abstract: The success of large-scale deep learning models in neuroscience is fundamentally constrained by severe data heterogeneity.
The paper introduces the Residual Patch Adapter (RPA), a lightweight module that aligns EEG and MEG signals with all patch tokens from an intermediate layer of a Vision Transformer, rather than a single global embedding. Ablation studies show that retaining the full set of patch tokens is crucial for effective alignment, while the CLS token adds little value. Using RPA, the authors achieve state‑of‑the‑art Top‑1 accuracies on THINGS‑EEG2 and THINGS‑MEG datasets, and demonstrate that the approach generalizes to other brain encoders and can be swapped with various neural network backbones.
The paper studies where to place task‑specific adapters in a vision transformer to balance storage growth and accuracy. Training all contiguous four‑block placements shows an inverted‑U accuracy curve, peaking at intermediate depths, while simple weight or activation metrics favor the deepest blocks. A neuroscience‑inspired method, LS‑B, uses frozen fMRI readouts of human visual areas to select blocks whose responses vary most across tasks, yielding backbone‑specific allocations that match or exceed the best placements found by search and use only 60% of the adapter storage while staying within 1.5 percentage points of full accuracy.
The paper introduces FAD‑ViT, a vision transformer that decouples spatial and feature axes of global attention, making feature‑axis attention the primary operator. Experiments on 23 medical imaging tasks show that shifting global attention from spatial to feature leaves from‑scratch performance unchanged but significantly boosts transfer learning, achieving up to three times the gain with fewer parameters and lower compute. The study demonstrates that prioritizing feature‑axis attention is a simple, cost‑effective design choice for improving medical image transfer.
arXiv:2610.15341v1 Announce Type: cross Abstract: The inherent variability of fMRI responses across trials encourages a generative view of the stimulus-to-response mapping. Prior generative encoders...
arXiv:2610.28411v1 Announce Type: cross Abstract: Deep learning models have become essential tools in medical imaging, yet training them end-to-end for diverse tasks remains challenging due to limite...
arXiv:2609.36391v1 Announce Type: cross Abstract: An ongoing challenge in sensory neuroscience is to characterize the feature dimensions encoded by cortical populations. Recent approaches probe featu...
Neurons++ is a visual‑stream framework that reconstructs both static images and dynamic videos from fMRI data. It builds on the earlier Neurons model by extending from video‑only to image‑and‑video reconstruction and replacing parallel task conditioning with a serial, cortex‑inspired decoding pipeline. The approach first pre‑trains on image‑fMRI pairs for stable spatial‑semantic initialization, then fine‑tunes separately for image and video tasks, organizing fMRI representations into spatial, semantic, language‑level, and temporal cues, and achieves state‑of‑the‑art video reconstruction with competitive image results while enabling neural‑to‑visual interpretability and ROI‑guided semantic modulation.