arXiv AI

An Approach to Simultaneous Acquisition of Real-Time MRI Video, EEG, and Surface EMG for Articulatory, Brain, and Muscle Activity During Speech Production

arXiv:2603. 04840v2 Announce Type: replace-cross Abstract: Speech production is a complex process spanning neural planning, motor control, muscle activation, and articulatory kinematics.

arXiv AI
Sep 25

SHINE: Sequential Hierarchical Integration Network for EEG and MEG

SHINE is a Sequential Hierarchical Integration Network designed to reconstruct speech envelope and Mel spectrogram from EEG and MEG recordings. It uses a residual sensor adapter, dilated-block states for temporal depth, and a target- and time-dependent gate to fuse hierarchical and attention-enhanced context predictions. Across two EEG and two MEG datasets, SHINE achieved the highest mean envelope and mean-Mel Pearson correlations among nine baseline methods and ranked second in the NeurIPS 2025 PNPL Competition’s speech-detection Extended Track.

By Xiran Xu, Yujie Yan, Songyi Li, Linze Zheng, Zifeng Zhang, Mochu Dong, Jing Chen
arXiv Machine Learning
6d ago

NeuroCodec: EEG-Guided Latent-Domain Generative Model for High-Fidelity Target Speaker Extraction

NeuroCodec is a novel generative framework that uses EEG signals to guide target speaker extraction in multi‑talker environments. It operates in the latent space of a neural audio codec, modeling EEG spatial dependencies with graph convolutions and fusing EEG and speech representations via attention and Mamba‑based temporal modeling. A six‑phase training curriculum and a causal streaming version enable high‑fidelity, 44.1 kHz audio reconstruction up to 20 kHz, outperforming baselines on Cocktail Party and KUL datasets and receiving higher perceptual scores in a listening test.

By Julian Silva, Jaliya Kodithuwakku, Navindu Sirimanna, Shaveen Herath, Wageesha N. Manamperi, Joshua P. Kulasingham
arXiv AI
Aug 20

Sensory Restoration via Brain-Computer Interfaces: A Scoping Review

This scoping review examines how brain‑computer interfaces (BCIs) can restore sensory and motor functions in people with severe neurological impairment. It introduces a unified 2×2 framework (invasiveness × signal direction) to map 31 key studies, highlighting that most evidence comes from invasive, efferent‑restoration approaches post‑2015. The paper outlines a roadmap toward closed‑loop, bidirectional restoration and identifies gaps in metric standardization, longitudinal data, and cross‑community collaboration.

By Xuan-The Tran
arXiv AI
Sep 28

Subject-Invariant Cross-Modal Decoding of Perceived Speech from Brain Recordings

The paper introduces the Subject-Invariant Cross-Modal Perceived Speech Decoding (SICMD) method, which fuses fMRI and MEG data to decode perceived speech from non‑invasive brain signals. Comprehensive experiments show that SICMD improves Top‑1, Top‑10, and Rankacc scores by over 10%, 10%, and 1.7% respectively, while cutting training costs by 88.8% and 60.5% compared to existing multi‑subject and intra‑subject approaches. Visualizations further confirm the method’s effectiveness.

By Aoke Zhang, Jing Chen