arXiv AI

SCRIBE: Scalable Contextual Representations of Interpretable Brain-Grounded Semantics

SCRIBE is a framework that uses a neuroscience-informed semantic space and an ensemble of large language models to generate interpretable, word-level semantic representations of arbitrary text. In a behavioral study with 921 participants, SCRIBE’s representations varied systematically with context and captured 75‑81% of the estimated human noise ceiling. When applied to brain activity recorded while participants listened to naturalistic stories, SCRIBE predicted neural responses in language-selective brain regions better than non-contextual representations and explained unique brain-predictive information beyond syntactic features.

arXiv AI
Oct 16

Decomposing Brain-LLM Alignment with Interpretable Semantic Representations

The study investigates how a large language model (LLaMA2‑7B) aligns with human brain activity during naturalistic language comprehension by using SCRIBE, an interpretable semantic representation of 65 neurocognitively grounded properties. Variance partitioning on fMRI data from seven participants reveals that shared variance between LLaMA2‑7B and SCRIBE dominates across the brain, yet both representations also contribute unique variance. Further analysis shows that concrete semantic properties correlate more strongly with shared variance, while abstract properties do not show increased unique variance, and ablation experiments demonstrate that multiple semantic dimensions cumulatively influence both shared and unique brain-predictive variance.

By Camila Kolling, Mariya Toneva
arXiv AI
Aug 20

Accurate Decoding of Natural Sentences from Non-Invasive Brain Recordings

Brain2Qwerty v2 is a model that decodes natural sentences from real‑time magnetoencephalography (MEG) recordings, achieving an average word error rate of 39% across 22,000 sentences typed by nine subjects. The model uses character, word, and sentence‑level representations and shows that decoding accuracy improves log‑linearly with more data, narrowing the gap to intracranial brain‑computer interfaces. AI contributes by replacing hand‑crafted event detection with deep learning, fine‑tuning large language models for semantic extraction, and employing AI agents to refine the decoding pipeline through automated code development.

By Mingfang Zhang, Jarod L\'evy, Cedric Rommel, J\'er\'emy Rapin, Corentin Bel, Julie Bonnaire, Daniel Nieto, Pierre Bourdillon, Svetlana Pinet, St\'ephane d'Ascoli, Thomas Moreau, Jean-R\'emi King
arXiv Computation and Language
Oct 12

Language Models Have Interpretable Narrative Mechanisms

The paper explores whether language models encode components that represent narrativity. By examining GPT‑2‑Small, Gemma‑2‑2B, and Llama‑3.1‑8B, the authors identify specific attention heads, MLP neurons, and sparse autoencoder features whose activations correlate with narrative properties. Ablating these components raises next‑token prediction loss more for high‑narrativity texts and diminishes narrativity in open‑ended generations, with the strongest effects seen in story completion and narrative question answering tasks.

By David Yiyang Liu, Aditya Joshi, Paul Dawson, Andrew Piper
arXiv Computation and Language
1d ago

Meaning Contracts: Local Semantic Geometry in Large Language Models

The paper investigates how semantic meaning is arranged locally within large language models. By using two controlled probes—one cross‑lingual and one refinement probe—on four decoder‑only LLMs, it finds that after removing dominant variance directions, sentence pairs rank by semantic relatedness, semantic factors cluster around an invariant core with structured variation, refinement tightens neighborhoods, and sentence‑level versus token‑local readouts reveal complementary geometric patterns. These results suggest that LLM semantics form local anisotropic neighborhoods that contract when meaning is refined.

By Andr\'es Daniel Godoy Ortiz