The paper introduces MD‑SigLIP, a margin‑regularized structured semantic alignment framework that directly aligns brain embeddings with text embeddings in a shared semantic space for retrieval‑based decoding. It builds on duplicate‑aware sigmoid contrastive learning and adds a listwise margin‑regularized term to enforce structured ranking constraints between positive semantic clusters and negative samples. Experiments show that this approach achieves state‑of‑the‑art retrieval performance in both full‑vocabulary and subset evaluation settings.
By Jiaqi Wang, Huawen Hu, Shu Zhang
arXiv:2609.10296v1 Announce Type: new
Abstract: Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phon...
By Gilad D. Landau, Dulhan Jayalath, Oiwi Parker Jones
arXiv:2607. 19394v1 Announce Type: cross Abstract: Generalizing across subjects remains challenging in invasive neural recordings because electrode configurations, anatomical structures, and neural signal patterns vary substantially across individuals.
By Ji-Hoon Heo, Aleksandra Joanna Wisniewska, Seo-Hyun Lee, Seong-Whan Lee
arXiv:2503. 06437v3 Announce Type: replace-cross Abstract: We present SEED (Semantic Evaluation for Visual Brain Decoding), a novel metric for evaluating the semantic decoding performance of visual brain decoding models.
By Juhyeon Park, Peter Yongho Kim, Jiook Cha, Shinjae Yoo, Taesup Moon
arXiv:2606. 15989v1 Announce Type: cross Abstract: Aligning neural activity across subjects offers the promise of discovering shared computational principles and generalizable decoders.
By Angeliki Papathanasiou, Jascha Achterberg, Thomas E. Nichols, Rui Ponte Costa
The study examined whether brain-language model alignment reflects shared computational mechanisms or merely stable lexical‑semantic correspondences. Using whole‑brain encoding across Mandarin, English, and French, transformer representations predicted activity in a distributed network that overlapped across languages and remained stable across layers. Contextual embeddings and measures of prediction or compression did not outperform static lexical embeddings, suggesting that alignment is robust but not informative about shared computational processes.
By Ni Yang, Rui He, Philipp Homan, Iris Sommer, Davide Staub, Wolfram Hinzen
arXiv:2606. 30319v1 Announce Type: cross Abstract: Modeling the bidirectional correspondence between external sensory stimuli and internal neural activity has emerged as a critical frontier in neuroscience.
By Haitao Wu, Qirui Zhang, Zhouheng Yao, Shangquan Sun, Qihao Zheng, Mianxin Liu, Chi Zhang, Wanli Ouyang, Chunfeng Song, Changqing Zhang, Jiamin Wu
The paper introduces NOEMA, a temporal convolutional decoder that aligns consecutive fMRI windows with multimodal embeddings (text, audio, and speech) of naturalistic speech stimuli. By leveraging these complementary representations, NOEMA can retrieve the correct speech segment from a pool of 2,000 candidates in the top 10 for over 42% of held‑out queries, outperforming baseline decoders even when the candidate set is expanded with thousands of unseen distractors. Ablation studies confirm that both temporal modeling and multimodal alignment are key to its performance.
By Corentin Caris, Louis Jalouzot, Nona Rajabi, G\'eraud Ilinca, Ahmed Hassayoune, Asilbek Muminov, Thadd\'ee Delebarre, Clara Fonteneau, Charles Maussion, Bertrand Thirion, Alexis Thual, Antoine Collas
arXiv:2601.21948v2 Announce Type: replace
Abstract: Neural visual decoding is a central problem in brain-computer interface research, aiming to reconstruct human visual perception and to elucidate th...
By Yang Du, Siyuan Dai, Yonghao Song, Paul M. Thompson, Haoteng Tang, Liang Zhan
arXiv:2608. 11656v1 Announce Type: new Abstract: Recent advances in EEG foundation models have demonstrated the potential of large-scale pretraining to enable generalizable neural decoding across subjects, recording environments, and datasets.
By Myeong-Ju Cho, Hye-Bin Shin, Seo-Hyun Lee, Seong-Whan Lee
DiscoVL introduces a disentangled cross‑modal representation learning framework for vision‑language models, combining orthogonal adversarial regularization with a multi‑branch low‑rank residual aligner to enhance bidirectional visual‑textual interaction. The method decomposes representations into subspaces, enabling cross‑modal feedback at each layer and preventing centroid collapse through an orthogonal adversarial triplet loss. Across 15 benchmarks, DiscoVL consistently outperforms state‑of‑the‑art approaches in base‑to‑novel generalization, cross‑dataset evaluation, and few‑shot learning.
By Mengping Dong, Jinbao Li, Fei Li
arXiv:2603. 17450v2 Announce Type: replace-cross Abstract: Sequential Recommendation (SR) in multimodal settings typically relies on small frozen pretrained encoders, which limits semantic capacity and prevents Collaborative Filtering (CF) signals from being fully integrated into item representations.
By Junyoung Kim, Woojoo Kim, Wonbin Kweon, Jaehyung Lim, Dongha Kim, Hwanjo Yu