arXiv Machine Learning

Recasting Visual Language Models as EEG Foundation Models for Clinical Neurodiagnostics

arXiv Machine Learning
Oct 8

Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding

The paper introduces BraVista, a visual-language framework that transforms multichannel EEG signals into structured images and uses instruction‑conditioned vision‑language models for unified multi‑task EEG decoding. By continuing post‑training of a general‑domain VLM, BraVista adapts to neural signals without large‑scale EEG‑specific pretraining, achieving strong performance across sleep staging, emotion recognition, cognitive workload classification, and abnormal EEG detection. Analyses show that the EEG‑to‑image representation is crucial and that the model’s performance degrades gradually with added noise, indicating reliance on genuine EEG information.

By Parastoo Azizeddin, Omid Sharafi, Maryam M. Shanechi
arXiv Machine Learning
Oct 28

OneEEG: A Unified Signal-Visual Multimodal Language Model with Task-Aware Mixture of Projectors for Multi-Task EEG Understanding

OneEEG is a unified signal‑visual multimodal language model designed for multi‑task EEG understanding. It jointly learns from numerical EEG data and visual EEG renderings, employing a task‑aware Mixture of Projectors to align diverse EEG features with a large language model’s embedding space. The authors introduce the OneEEG‑Instruct dataset for instruction tuning and the OneEEG‑Bench benchmark to evaluate performance, demonstrating that existing multimodal LLMs struggle with EEG inputs while OneEEG outperforms them across various tasks.

By Jiacheng Zhang, Xueyi Zhang, Xin Lu, Siqi Cai, Haizhou Li
arXiv AI
Aug 28

EEG-to-Report: An Annotation and Feature-Text Framework for Training Language Models on Clinical EEG

EEG-to-Report is a browser-based annotation and feature‑text framework that links routine EEG review with the creation of AI‑ready datasets. It ingests multi‑format EEG data, standardizes channels, and provides an interactive viewer with a multimodal annotation layer that combines typed text and transcribed voice notes. For each annotated segment, a feature extraction engine computes standardized spectral, temporal, entropy, Hjorth, connectivity, and spike‑related descriptors, stored alongside clinical descriptions in a portable JSON schema, producing aligned feature‑text pairs for training multimodal EEG‑language models. The framework also includes an auto‑report module that uses an ensemble of convolutional networks and a large language model to draft clinical narratives for neurologist review, thereby streamlining annotation workflows and enabling editable draft reports.

By Xuan-The Tran, Le Trung Kien Nguyen
arXiv Machine Learning
Aug 4

EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models

arXiv:2601. 17883v3 Announce Type: replace Abstract: Electroencephalography (EEG) foundation models (FMs) have recently emerged as a promising paradigm for brain-computer interfaces, aiming to learn transferable neural representations from large-scale heterogeneous recordings.

By Dingkun Liu, Yuheng Chen, Zhu Chen, Zhenyao Cui, Yaozhi Wen, Jiayu An, Jingwei Luo, Dongrui Wu
arXiv AI
Sep 2

EEG-AS: Instance-Level Foundation Model Selection for EEG Foundation Models via Behavior Reconstruction

EEG-AS is an instance-level algorithm selection framework designed for EEG foundation models. It characterizes each EEG instance using latent embeddings, handcrafted neurophysiological features, and an anchor foundation model, then learns to reconstruct the behaviors of other foundation models from privileged prediction tokens. During inference, EEG-AS estimates these behaviors without running the full model portfolio, enabling efficient selection among seven EEG foundation models and significantly reducing the performance gap between the single best solver and the oracle upper bound across seven public EEG benchmarks.

By Yunzhen Zhang, Ruoxi Piao, Hasan Onur Keles, Mustafa Misir