arXiv Machine Learning June 19, 2026 By Masato Takagi, Masaya Kawamura, Reo Shimizu, Yuma Shirahata

Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations

Read the original on arXiv Machine Learning →

arXiv:2606. 19951v1 Announce Type: cross Abstract: Mean opinion score (MOS) prediction models are widely used as proxy metrics in text-to-speech (TTS) research, yet their ability to capture quality differences beyond acoustic fidelity remains unclear.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 11

Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions

arXiv:2608. 09930v1 Announce Type: cross Abstract: Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Models (Audio-LLM) judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive.

By Oluwanifemi Bamgbose, Simon Rosen, Jash Shah, Lindsay Devon Brin, Hoang H Nguyen, Anke Koelzer, Rachel Hansen, Tara Bogavelli, Fanny Riols

llms multimodal benchmarks

arXiv AI

Aug 24

DAMOS: Learning Distortion-Aware Speech Quality Assessment through Explicit Distortion Localization

arXiv:2608.21176v1 Announce Type: cross Abstract: Automatic speech quality assessment aims to predict Mean Opinion Scores (MOS) consistent with human subjective perception and is essential for evalua...

By Naiyuan Li, Li Dong, Diqun Yan

benchmarks

arXiv Computation and Language

Sep 1

Why We Need Speech to Evaluate Speech Translation

arXiv:2605.28227v2 Announce Type: replace Abstract: Speech translation models are increasingly capable of preserving speech-specific information (e.g., speaker gender, prosody, and emphasis), yet eva...

By Maike Z\"ufle, Danni Liu, Vil\'em Zouhar, Jan Niehues

benchmarks

arXiv AI

Jun 16

NVMOS: Non-Verbal Vocalization Quality Assessment in Speech

arXiv:2606. 15888v1 Announce Type: cross Abstract: Non-verbal vocalizations (NVs), such as laughter, sighs, and coughs, are important acoustic cues for emotion and intent.

By Jialong Mai, Jinxin Ji, Xiaofen Xing, Wencui Liu, Xiangmin Xu

llms multimodal

arXiv Machine Learning

Jun 19

PASQA: Pitch-Accent-Focused Speech Quality Assessment Model Trained on Synthetic Speech with Accent Errors

arXiv:2606. 20137v1 Announce Type: cross Abstract: Existing mean opinion score (MOS) prediction models typically predict utterance-level naturalness MOS and can be insensitive to localized pitch-accent errors.

By Masaya Kawamura, Yuma Shirahata, Kentaro Mitsui, Reo Shimizu

multimodal

arXiv AI

4d ago

HEAR: Real Voices, Real Bias: A Large-Scale Human-Recorded, Demographically Diverse Benchmark for Audio Language Models

arXiv:2609.35952v1 Announce Type: cross Abstract: We introduce HEAR (Human-recorded Evaluation of Audio-LLM bias by Real speakers), a large-scale, ecologically valid benchmark comprising 87k real hum...

By Shen Yan, Duc Le, Irina-Elena Veliche

llms nlp benchmarks safety

One email a morning, machine-written

Related stories

Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions

DAMOS: Learning Distortion-Aware Speech Quality Assessment through Explicit Distortion Localization

Why We Need Speech to Evaluate Speech Translation

NVMOS: Non-Verbal Vocalization Quality Assessment in Speech

PASQA: Pitch-Accent-Focused Speech Quality Assessment Model Trained on Synthetic Speech with Accent Errors

HEAR: Real Voices, Real Bias: A Large-Scale Human-Recorded, Demographically Diverse Benchmark for Audio Language Models