arXiv Machine Learning
Recasting Visual Language Models as EEG Foundation Models for Clinical Neurodiagnostics
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Oct 8
Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding
The paper introduces BraVista, a visual-language framework that transforms multichannel EEG signals into structured images and uses instruction‑conditioned vision‑language models for unified multi‑task EEG decoding. By continuing post‑training of a general‑domain VLM, BraVista adapts to neural signals without large‑scale EEG‑specific pretraining, achieving strong performance across sleep staging, emotion recognition, cognitive workload classification, and abnormal EEG detection. Analyses show that the EEG‑to‑image representation is crucial and that the model’s performance degrades gradually with added noise, indicating reliance on genuine EEG information.
arXiv AI
Aug 14
Can Generalist Vision Language Models (VLMs) Rival Specialist Medical VLMs? Benchmarking and Strategic Insights
arXiv:2506. 17337v5 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) have shown promise in automating image diagnosis and interpretation in clinical settings.
arXiv Machine Learning
Oct 28
OneEEG: A Unified Signal-Visual Multimodal Language Model with Task-Aware Mixture of Projectors for Multi-Task EEG Understanding
OneEEG is a unified signal‑visual multimodal language model designed for multi‑task EEG understanding. It jointly learns from numerical EEG data and visual EEG renderings, employing a task‑aware Mixture of Projectors to align diverse EEG features with a large language model’s embedding space. The authors introduce the OneEEG‑Instruct dataset for instruction tuning and the OneEEG‑Bench benchmark to evaluate performance, demonstrating that existing multimodal LLMs struggle with EEG inputs while OneEEG outperforms them across various tasks.
arXiv Computer Vision
Sep 16
BrainFocus: EEG-Guided ROI Selection for Efficient Vision-Language Models
arXiv:2609.17443v1 Announce Type: new Abstract: Vision-language models (VLMs) achieve strong visual question answering (VQA) performance, but processing large cluttered images is computationally expe...
arXiv AI
Aug 28
EEG-to-Report: An Annotation and Feature-Text Framework for Training Language Models on Clinical EEG
EEG-to-Report is a browser-based annotation and feature‑text framework that links routine EEG review with the creation of AI‑ready datasets. It ingests multi‑format EEG data, standardizes channels, and provides an interactive viewer with a multimodal annotation layer that combines typed text and transcribed voice notes. For each annotated segment, a feature extraction engine computes standardized spectral, temporal, entropy, Hjorth, connectivity, and spike‑related descriptors, stored alongside clinical descriptions in a portable JSON schema, producing aligned feature‑text pairs for training multimodal EEG‑language models. The framework also includes an auto‑report module that uses an ensemble of convolutional networks and a large language model to draft clinical narratives for neurologist review, thereby streamlining annotation workflows and enabling editable draft reports.
arXiv AI
Jul 24
MSBraM: A Multi-scale Self-supervised Brain Foundation Model for Hierarchical EEG Dynamics Learning
arXiv:2607. 21402v1 Announce Type: new Abstract: Self-supervised foundation models have recently shown strong potential for electroencephalogram (EEG)-based analysis.