Brain-to-image retrieval seeks to identify the visual stimulus that elicited a non-invasive neural response. Candidate images are typically represented by pretrained vision models, whose internal repr...
arXiv:2609.36971v1 Announce Type: new Abstract: Cross-subject EEG-to-image retrieval requires a neural represen- tation trained on source subjects to remain aligned with a visual embedding space for...
arXiv:2606. 00121v1 Announce Type: cross Abstract: Reconstructing visual stimuli from brain recordings has been a meaningful and challenging task in brain decoding.
The paper presents a reproducible single‑subject baseline for reconstructing visual stimuli from EEG using a temporal‑spatial convolutional encoder that maps averaged EEG signals to 512‑dimensional ViT-B/32 image features. On the THINGS‑EEG2 dataset, the model achieves 12.83%, 39.17%, and 58.00% image recall at ranks 1, 5, and 10, respectively, outperforming analytical chance levels. The study also shows that performance drops sharply when applying a model trained on one subject to others, and that direct conditional generators without external visual weights produce noise‑dominated outputs, indicating that only coarse semantic decoding is feasible under the tested protocol.
The paper introduces the Residual Patch Adapter (RPA), a lightweight module that aligns EEG and MEG signals with all patch tokens from an intermediate layer of a Vision Transformer, rather than a single global embedding. Ablation studies show that retaining the full set of patch tokens is crucial for effective alignment, while the CLS token adds little value. Using RPA, the authors achieve state‑of‑the‑art Top‑1 accuracies on THINGS‑EEG2 and THINGS‑MEG datasets, and demonstrate that the approach generalizes to other brain encoders and can be swapped with various neural network backbones.
The paper introduces NEAR, a neural-anchor-based retrieval framework that improves brain-to-image retrieval when only a few neural trials are available. By treating a high‑repetition center as an anchor, NEAR uses a denoiser to pull noisy queries toward the anchor and a small network to predict pseudo anchors for candidate images, thereby aligning both neural and visual representations. Experiments on EEG, MEG, and fMRI datasets show consistent gains, including a 5.7–9.3 percentage point improvement in 200‑way Top‑1 accuracy on THINGS‑EEG2 with only one or four repetitions.
NeuroBridge is a self‑supervised framework for EEG‑to‑image decoding that uses Cognitive Prior Augmentation (CPA) to introduce perceptual variability and a Shared Semantic Projector (SSP) to align EEG and image features in a shared semantic space. The method achieves significant gains over prior work, improving top‑1 accuracy by 12.3% and top‑5 accuracy by 10.2% on a 200‑way zero‑shot retrieval task in intra‑subject experiments. Extensive tests show the approach is robust, scalable, and effective for neural visual decoding.
arXiv:2610.09440v1 Announce Type: new Abstract: Pre-trained vision encoders contain layer-wise visual representations that differ in spatial granularity, semantic abstraction, and sensitivity to loca...
arXiv:2608. 20810v1 Announce Type: cross Abstract: Multimodal information systems increasingly route generated visual content back through the same vision-language index that informed its production, so the output must remain retrievable by the queries it was meant to serve.
ProtoEEG introduces a prototype‑mediated framework for aligning EEG signals with visual data, using a shared 64‑prototype codebook to generate structured representations for retrieval and image generation. The method employs prototype contrast, distribution alignment, instance anchors, and diversity regularization to train the system, achieving state‑of‑the‑art performance on THINGS‑EEG2 and EEG‑ImageNet benchmarks. Ablation studies confirm that prototype contrast and identity preservation are critical for high accuracy, while prototype analyses reveal distributed assignments across the codebook.
The paper studies where to place task‑specific adapters in a vision transformer to balance storage growth and accuracy. Training all contiguous four‑block placements shows an inverted‑U accuracy curve, peaking at intermediate depths, while simple weight or activation metrics favor the deepest blocks. A neuroscience‑inspired method, LS‑B, uses frozen fMRI readouts of human visual areas to select blocks whose responses vary most across tasks, yielding backbone‑specific allocations that match or exceed the best placements found by search and use only 60% of the adapter storage while staying within 1.5 percentage points of full accuracy.
SCORE: Subject Coordinate Recovery for Label-Free Cross-Subject EEG-to-Image Retrieval proposes a new framework that aligns EEG signals from different subjects into a common image space without requiring labeled calibration data. By training on source subjects and estimating an orthogonal transformation at deployment, SCORE recovers target EEG coordinates and selects reliable EEG-image landmarks through hubness-corrected matching. The method achieves state‑of‑the‑art Top‑1/Top‑5 accuracy on two public benchmarks, outperforming existing baselines by significant margins.