arXiv Machine Learning

EEG-Captioner: Grounding EEG-to-Language Generation in Fundamental Signal Properties

EEG-Captioner introduces a new EEG-to-language framework that grounds language generation in measurable EEG signal properties. The authors built a large dataset of 160,781 EEG segments annotated with question‑answer pairs targeting spectral, spatial, temporal, and lateral properties, and paired each segment with a feature‑grounded caption. Their Dual‑Domain EEG Encoder jointly models temporal waveforms and spectral‑spatial representations, achieving a macro‑averaged balanced accuracy of 88.81% on signal‑level QA and improving caption factual consistency from 82.06% to 83.46% compared to caption‑only models.

arXiv Machine Learning
Oct 16

Semantic Alignment of EEG and Text: A Contrastive Learning Framework for Decoding Imagined Speech

The paper introduces a multimodal contrastive learning framework that aligns EEG recordings of imagined speech with language representations from a frozen text encoder. By training an EEG encoder with a composite objective that balances semantic reweighting and class discrimination, the method preserves separability among 39 intent categories while embedding EEG signals into a language embedding space. Experiments on the Chisco dataset show comparable decoding performance to a task‑optimized classifier, and additional metrics such as MaxCosine and semantic axes demonstrate intrinsic reliability and broader semantic structure beyond fixed categories.

By Shahryar G. Namdari, Francesco Iacomi, Riccardo Barbieri, Luca Mainardi
arXiv Machine Learning
Jul 22

Is EEG-to-Text Feasible in Real-World Scenarios? An In-Depth Analysis Using a Neuropsychology-Inspired Benchmark

arXiv:2607. 18749v1 Announce Type: new Abstract: Translating brain signals into text could restore communication for people with severe paralysis, yet practically usable systems to date rely on invasive electrocorticography (ECoG).

By Zihan Zhang (Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology), Yu Bao (Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, Shanghai Innovation Institute), Xiao Ding (Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology), Tianyi Jiang (State Key Laboratory for Novel Software Technology, Nanjing University), Kai Xiong (Zhongguancun Laboratory)
arXiv AI
Aug 28

EEG-to-Report: An Annotation and Feature-Text Framework for Training Language Models on Clinical EEG

EEG-to-Report is a browser-based annotation and feature‑text framework that links routine EEG review with the creation of AI‑ready datasets. It ingests multi‑format EEG data, standardizes channels, and provides an interactive viewer with a multimodal annotation layer that combines typed text and transcribed voice notes. For each annotated segment, a feature extraction engine computes standardized spectral, temporal, entropy, Hjorth, connectivity, and spike‑related descriptors, stored alongside clinical descriptions in a portable JSON schema, producing aligned feature‑text pairs for training multimodal EEG‑language models. The framework also includes an auto‑report module that uses an ensemble of convolutional networks and a large language model to draft clinical narratives for neurologist review, thereby streamlining annotation workflows and enabling editable draft reports.

By Xuan-The Tran, Le Trung Kien Nguyen
arXiv AI
Aug 6

BrainBench: Benchmarking Large Language Models for Comprehensive EEG Understanding

arXiv:2608. 04156v1 Announce Type: new Abstract: Electroencephalography (EEG) analysis extends beyond assigning predefined labels to recordings; it requires workflows connecting natural-language instructions, signal processing, quantitative evidence, and scientific interpretation.

By Yangxuan Zhou, Sha Zhao, Yuning Chen, Chen Wu, Jiquan Wang, Shijian Li, Gang Pan
arXiv Machine Learning
Oct 8

Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding

The paper introduces BraVista, a visual-language framework that transforms multichannel EEG signals into structured images and uses instruction‑conditioned vision‑language models for unified multi‑task EEG decoding. By continuing post‑training of a general‑domain VLM, BraVista adapts to neural signals without large‑scale EEG‑specific pretraining, achieving strong performance across sleep staging, emotion recognition, cognitive workload classification, and abnormal EEG detection. Analyses show that the EEG‑to‑image representation is crucial and that the model’s performance degrades gradually with added noise, indicating reliance on genuine EEG information.

By Parastoo Azizeddin, Omid Sharafi, Maryam M. Shanechi
arXiv AI
Jun 11

Brain-IT-VQA: From Brain Signals to Answers

arXiv:2605. 29588v2 Announce Type: replace-cross Abstract: Decoding visual content from fMRI signals recorded while a person views images, and specifically answering questions about the seen images, is a long-standing challenge.

By Roman Beliy, Matias Cosarinsky, Oliver Heinimann, Navve Wasserman, Michal Irani
arXiv AI
Sep 7

ProCA: Progressive Contrastive Alignment for Robust EEG Visual Decoding

ProCA: Progressive Contrastive Alignment for Robust EEG Visual Decoding introduces a model‑agnostic framework that adaptively aligns EEG signals with visual semantics. It replaces fixed visual or textual anchors with EEG‑aware class‑level contrastive supervision and employs structure‑consistent interpolation to preserve channel‑wise and temporal importance. Across multiple evaluation settings—including subject‑dependent, subject‑independent, strict cross‑subject transfer, and continual adaptation—ProCA delivers significant performance gains, achieving relative Top‑1 improvements ranging from 7.4% to 28.1%.

By Kanglei Zhou, Chunyan Lan, Dongyang Li, Jun Zhu, Liyuan Wang