arXiv AI October 19, 2026 By Mingyue Huo, Shivam Mehta, Bhavin Jawade, Yinghong Lan, Haoqi Li

SpeechCritic: Learning a Diagnostic Speech Judge from Limited Human Preferences

SpeechCritic learns a diagnostic speech judge from only about 300 human-labeled comparisons, calibrating a frontier audio-language model with non-binding hints that select acoustic measurements agreeing with human judgments. The approach improves dimension-level agreement by 6.3 points and reduces mismatch in Tie rates by 10.4 points, and training a 7B judge with SFT, OPD, and RL yields distinct behaviors, with RL producing more specific rationales. The method works across language pairs, demonstrating a path from limited human preferences to a diagnostic speech judge.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Sep 15

The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speech

arXiv:2609.13150v1 Announce Type: cross Abstract: Reference-free quality predictors such as UTMOS, DNSMOS and SCOREQ are the de facto automatic evaluators for text-to-speech (TTS) and are increasingl...

By Antonis Asonitis, Juan Pablo Zuluaga Gomez, Francesco Verdini, Aref Farhadipour, Marzieh Razavi, Pierre-Edouard Honnet, Vijeta Avijeet

multimodal

arXiv AI

Aug 11

Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions

arXiv:2608. 09930v1 Announce Type: cross Abstract: Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Models (Audio-LLM) judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive.

By Oluwanifemi Bamgbose, Simon Rosen, Jash Shah, Lindsay Devon Brin, Hoang H Nguyen, Anke Koelzer, Rachel Hansen, Tara Bogavelli, Fanny Riols

llms multimodal benchmarks

arXiv Machine Learning

Aug 28

Interpretable, Fairly Evaluated Automated L2 Speaking Assessment that Beats the Single-Human Ceiling and Why Pause Encoding Does Not Change LLM Fluency Scores

The paper presents an interpretable, fair, and accurately benchmarked automated system for assessing second‑language English speaking. Using a hybrid of feature‑based speech‑timing metrics and a large language model (LLM) fluency judgment, the system achieves a Spearman correlation of 0.818 with the ICNALE Global Rating Archive, outperforming 81 % of trained human raters. A controlled study shows that encoding pauses into the LLM prompt does not meaningfully affect fluency scores, indicating that the system’s fluency signal derives from measurable speech‑timing features.

By Eichi Uehara

llms safety

arXiv Computation and Language

Sep 1

When Does Predictor-Based RL Align with Human Perception? A Study of Subjective Rewards in Codec-Based Speech Language Models

arXiv:2608.31035v1 Announce Type: new Abstract: Codec-based text-to-speech (TTS) models make language-model post-training applicable to speech generation, but it remains unclear when learned perceptu...

By Joonyong Park, Jerry Li

llms rag reinforcement-learning multimodal safety

arXiv Computation and Language

Oct 5

CoLMbo-SV: A Grounded Language Model for Explainable Speaker Verification

arXiv:2609.33212v2 Announce Type: replace Abstract: Speaker verification systems achieve high accuracy but provide little account of the acoustic evidence behind their judgments. Making these systems...

By Massa Baali, Sarthak Bisht, Ziyue Qiu, Joseph Konan, Rita Singh, Bhiksha Raj

llms fine-tuning

Hugging Face Trending Papers

Jun 23

ParaPairAudioBench: Paralinguistic Pairwise Audio Benchmark for LALM-as-a-Judge

Large Audio-Language Models (LALMs) have been widely used as judge models for the automatic evaluation of generated speech. However, prior approaches predominantly focus on holistic naturalness, leaving fine-grained paralinguistic distinctions underexplored.

llms benchmarks