arXiv Machine Learning

MindAU: EEG-Conditioned Facial Action Unit Editing via Dual-Stream Manifold Alignment

arXiv:2607. 00410v1 Announce Type: cross Abstract: Recent brain decoding studies have made substantial progress in reconstructing externally perceived visual content from neural signals.

arXiv Computer Vision
Oct 7

RPA: Residual Patch-Token Adapter for Image Retrieval from EEG and MEG

The paper introduces the Residual Patch Adapter (RPA), a lightweight module that aligns EEG and MEG signals with all patch tokens from an intermediate layer of a Vision Transformer, rather than a single global embedding. Ablation studies show that retaining the full set of patch tokens is crucial for effective alignment, while the CLS token adds little value. Using RPA, the authors achieve state‑of‑the‑art Top‑1 accuracies on THINGS‑EEG2 and THINGS‑MEG datasets, and demonstrate that the approach generalizes to other brain encoders and can be swapped with various neural network backbones.

By Yuhui Jin, Yonghao Song, Bingchuan Liu
arXiv Computer Vision
Sep 4

Occlusion-Robust Multimodal Emotion Recognition in VR via Fusion of Facial Images and EMG

The paper presents a method for emotion recognition in virtual reality where head‑mounted displays occlude the upper face. By fusing lower‑face video with electromyography (EMG) signals from the occluded upper face, the authors achieve a 51% macro‑F1 score across seven emotional categories, outperforming image‑only and EMG‑only baselines. A new synchronized multimodal dataset from 20 participants is introduced and will be shared under an ethical‑use agreement.

By Birgit Nierula, Karam Tomotaki-Dawoud, Mert Akguel, Mustafa Tevfik Lafci, David Przewozny, Anna Hilsmann, Peter Eisert, Sebastian Bosse
arXiv AI
Jun 2

UF-AMA: A unified framework for cross-domain emotion recognition via adaptive multimodal alignment

arXiv:2606. 00170v1 Announce Type: cross Abstract: In recent years, emotion recognition based on physiological signals such as electroencephalogram (EEG) has gained considerable attention, as internal physiological data offer greater objectivity and reliability compared to external behavioral data like facial expressions.

By Zheng Wang, Shuo Wang, Junhong Wang
arXiv AI
Sep 28

LEAD: An EEG Foundation Model for Alzheimer's Disease Detection

LEAD is a gated temporal‑spatial Transformer foundation model designed for EEG‑based Alzheimer's disease detection. It was trained on the world’s largest EEG‑AD corpus of 2,238 subjects and uses a subject‑regularized strategy and medical contrastive learning across 13 datasets. LEAD outperforms existing methods on five downstream AD datasets, achieving the best average ranking across 20 evaluations.

By Yihe Wang, Nan Huang, Nadia Mammone, Marco Cecchi, Xiang Zhang
arXiv AI
Aug 28

EEG-to-Report: An Annotation and Feature-Text Framework for Training Language Models on Clinical EEG

EEG-to-Report is a browser-based annotation and feature‑text framework that links routine EEG review with the creation of AI‑ready datasets. It ingests multi‑format EEG data, standardizes channels, and provides an interactive viewer with a multimodal annotation layer that combines typed text and transcribed voice notes. For each annotated segment, a feature extraction engine computes standardized spectral, temporal, entropy, Hjorth, connectivity, and spike‑related descriptors, stored alongside clinical descriptions in a portable JSON schema, producing aligned feature‑text pairs for training multimodal EEG‑language models. The framework also includes an auto‑report module that uses an ensemble of convolutional networks and a large language model to draft clinical narratives for neurologist review, thereby streamlining annotation workflows and enabling editable draft reports.

By Xuan-The Tran, Le Trung Kien Nguyen
arXiv AI
Sep 28

Neural State Prediction: Obstructing Shortcut Learning in EEG Foundation Models

Neural State Prediction (NSP) is a latent‑predictive framework designed to curb shortcut learning in EEG foundation models. By using a target encoder updated with an exponential moving average, identity residualization, and topology‑separated context, NSP constrains both the prediction target and the available context. Trained on 2.2 million EEG segments, NSP outperforms baselines on 14 datasets in the EEG‑FM‑Bench, achieving 63.94 % macro balanced accuracy.

By Kieren Yu, Ziyang Liu, Chang Huang, Jintai Chen, Kaishun Wu