The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper introduces Open‑Vocabulary Mutual Information (OVMI), an information‑theoretic metric that quantifies how much of a user’s intended speech a speech brain‑computer interface (BCI) can convey relative to a reference word distribution. OVMI enables comparison of systems that use different vocabularies, recording methods, and datasets, revealing that conventional metrics like accuracy and word error rate can overstate performance. Using OVMI, the authors compare existing speech BCI systems, expose trade‑offs between vocabulary coverage and decoding accuracy, and show that optimizing vocabulary selection for OVMI can improve accuracy by up to 16.3% across three speech domains.
The article surveys brain‑to‑language decoding, covering tasks, neural signals, methods, and evaluation across invasive and non‑invasive modalities. It traces the field’s evolution from constrained recognition to text generation, streaming speech, and facial animation, linking tasks to neural populations and decoder representations. The review highlights complementary decoding targets, shared representations, and the growing importance of calibration, feedback, and user control for online communication, while proposing a five‑level future trajectory toward bidirectional cognitive exchange.
arXiv:2609.27650v4 Announce Type: replace Abstract: Brain-to-language decoding translates neural activity associated with language production, internal speech and perception into linguistic or expres...
arXiv:2609.36736v1 Announce Type: cross Abstract: Speech brain-computer interfaces (BCIs) aim to restore communication by transforming neural activity related to speech, language, or communicative in...
arXiv:2607. 18749v1 Announce Type: new Abstract: Translating brain signals into text could restore communication for people with severe paralysis, yet practically usable systems to date rely on invasive electrocorticography (ECoG).
EEG-Captioner introduces a new EEG-to-language framework that grounds language generation in measurable EEG signal properties. The authors built a large dataset of 160,781 EEG segments annotated with question‑answer pairs targeting spectral, spatial, temporal, and lateral properties, and paired each segment with a feature‑grounded caption. Their Dual‑Domain EEG Encoder jointly models temporal waveforms and spectral‑spatial representations, achieving a macro‑averaged balanced accuracy of 88.81% on signal‑level QA and improving caption factual consistency from 82.06% to 83.46% compared to caption‑only models.