The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper presents a method for inferring a speaker’s first language (L1) from their second‑language English (L2) by using token probability scores from large language models (LLMs). By treating each text as a continuation of a prompt that specifies a candidate L1, the authors identify the L1 with the highest probability score. Experiments with Llama 3.1 8B and 70B on TOEFL9, Write and Improve 2024, and BULATS‑L1 datasets show that token‑probability scoring outperforms direct classification, with the 70B model achieving state‑of‑the‑art results across all datasets.
The paper introduces a native-reference phone‑class geometry that measures second‑language pronunciation deviation without needing pronunciation labels, read‑aloud prompts, or matched native recordings. By averaging self‑supervised representations for each phone‑class in a native speech corpus and applying singular value decomposition, the authors create a compact coordinate system. Projecting L2 utterances into this space, they find that distances to native coordinates correlate negatively with holistic speaking proficiency and pronunciation quality, indicating the geometry captures relevant acoustic‑phonetic information for spontaneous L2 speech.
The paper presents a method to automatically identify utterances in child-centered daylong audio recordings that can be reliably transcribed by modern ASR systems, enabling accurate transcription of a substantial portion of the speech. On four English corpora, the approach achieves a median WER of 0% and a mean WER of 16% when transcribing 30% of the total speech, compared to a median WER of 52% when transcribing all speech. Word frequency distributions from the automatic transcripts correlate strongly with manual annotations (r = 0.94 overall, r = 0.99 for frequent words).
The paper introduces a lightweight approach to detect code‑switching (CS) directly from speech by leveraging language‑selective hidden‑state dimensions in pretrained speech encoders. Using monolingual recordings to identify these dimensions, the method aggregates frame‑level activations to predict CS presence, achieving a mean F1 score of 0.798 on the CS‑FLEURS dataset with the Whisper‑small encoder. This demonstrates that pretrained speech models inherently encode CS information, offering a practical tool for expanding multilingual corpora.
arXiv:2608. 09930v1 Announce Type: cross Abstract: Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Models (Audio-LLM) judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive.
SpeechCritic learns a diagnostic speech judge from only about 300 human-labeled comparisons, calibrating a frontier audio-language model with non-binding hints that select acoustic measurements agreeing with human judgments. The approach improves dimension-level agreement by 6.3 points and reduces mismatch in Tie rates by 10.4 points, and training a 7B judge with SFT, OPD, and RL yields distinct behaviors, with RL producing more specific rationales. The method works across language pairs, demonstrating a path from limited human preferences to a diagnostic speech judge.