arXiv Computation and Language
Do speech foundation models really learn words?
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
3d ago
Interleaved Speech Language Models Latently Work In Text
arXiv:2606.22473v2 Announce Type: replace-cross Abstract: Speech language models (SLMs) increasingly combine speech and text, often by interleaving their tokens within a single sequence. Yet how thes...
arXiv AI
Jun 11
Pretrained self-supervised speech models can recognize unseen consonants
arXiv:2606. 11542v1 Announce Type: cross Abstract: Modern pretrained self-supervised automatic speech recognition models are trained on large-scale audio data to encode speech into contextualized representations.
arXiv AI
Aug 25
Multi-Task Learning for Non-Canonical Phoneme Recognition via Articulatory Feature Decomposition
The paper proposes a linguistically structured multi‑task learning framework for recognizing non‑canonical phonemes by decomposing phoneme prediction into articulatory feature dimensions such as manner, place, and voicing. A hierarchical architecture with task‑specific heads and a cross‑attention fusion module is combined with semi‑supervised Momentum Pseudo‑Labeling and a cascaded training strategy that gradually introduces articulatory tasks. Experiments on the L2‑ARCTIC dataset demonstrate significant improvements over baseline models and produce interpretable error patterns aligned with phonological feature structure.
arXiv Computation and Language
Sep 4
Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe
The paper introduces an Encoding Probe that reconstructs language model representations using interpretable features, addressing limitations of traditional decoding probes such as incomparable feature contributions and correlation effects. It evaluates this approach on text and speech transformer models, examining features from acoustics, phonetics, syntax, lexicon, and speaker identity. Findings reveal that speaker-related effects vary with training objectives and datasets, while syntactic and lexical features independently contribute to reconstruction, offering a complementary perspective on model interpretation.
arXiv Computation and Language
Sep 1
A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification
arXiv:2512.07571v3 Announce Type: replace Abstract: This paper presents a simple method that allows to easily enhance textual pre-trained large language models with speech information, when fine-tune...
arXiv AI
Jun 16
LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization
arXiv:2506. 16738v2 Announce Type: replace-cross Abstract: With the rapid progress of speech language models (SLMs), discrete speech tokens have emerged as a core interface between speech and text, enabling unified modeling across modalities.