arXiv AI
Jun 2
Domain-Shift-Aware Conformal Prediction for Large Language Models
arXiv:2510. 05566v2 Announce Type: replace-cross Abstract: Large language models have achieved impressive performance across diverse tasks.
ConformalMOS: Uncertainty-Aware MOS Prediction with Conformal Intervals and Ordinal Modeling
ConformalMOS is a framework that enhances Mean Opinion Score (MOS) prediction for synthetic speech by adding conformal prediction to generate statistically valid prediction intervals with guaranteed coverage under exchangeability, while also providing point estimates. The method models MOS labels in an ordinal-aware way by converting one-hot targets into Gaussian-smoothed distributions, which preserves score ordering and improves uncertainty calibration, especially when softmax confidence is overconfident on out-of-distribution speech. On BVCC datasets, ConformalMOS achieves competitive accuracy (MSE = 0.08) and empirically validated coverage, thereby improving reliability for real-world TTS and VC deployment.
Related stories
arXiv Machine Learning
Jun 19
Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations
arXiv:2606. 19951v1 Announce Type: cross Abstract: Mean opinion score (MOS) prediction models are widely used as proxy metrics in text-to-speech (TTS) research, yet their ability to capture quality differences beyond acoustic fidelity remains unclear.
arXiv AI
Sep 15
CAL-MOS: Bridging Layers with Adapters for Robust MOS Prediction Across Speech Foundation Models
arXiv:2609.14956v1 Announce Type: cross Abstract: Speech Quality Assessment (SQA) is essential for modern speech technologies, and recent non-intrusive SQA predictors increasingly rely on Speech Foun...
arXiv AI
Jun 19
PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasets
arXiv:2606. 19597v1 Announce Type: cross Abstract: Mean opinion scores (MOS) are widely used for speech quality assessment, yet scalar labels are sensitive to rater variability and listening test differences.
arXiv AI
Aug 24
DAMOS: Learning Distortion-Aware Speech Quality Assessment through Explicit Distortion Localization
arXiv:2608.21176v1 Announce Type: cross Abstract: Automatic speech quality assessment aims to predict Mean Opinion Scores (MOS) consistent with human subjective perception and is essential for evalua...
arXiv AI
Sep 28
Audio LLMs Know When They Can't Hear You
The paper investigates whether audio large language models (Audio LLMs) can detect when their own transcriptions are unreliable. It finds that the models are poor at self-assessment and that existing methods offer limited detection. By leveraging audio-encoder representations, the authors develop a lightweight predictor that accurately flags unreliable transcriptions and can prompt user clarification without altering the underlying model.
Hugging Face Trending Papers
Jun 17
PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasets
Mean opinion scores (MOS) are widely used for speech quality assessment, yet scalar labels are sensitive to rater variability and listening test differences. This introduces labeling noise, which limits the reliability of MOS prediction.
arXiv AI
Aug 11
Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions
arXiv:2608. 09930v1 Announce Type: cross Abstract: Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Models (Audio-LLM) judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive.
arXiv Computation and Language
Oct 16
SITAR: A Situational Distortion Metric for Automatic Speech Recognition Evaluation
The paper introduces SITAR, a situational distortion metric for evaluating automatic speech recognition (ASR) systems. SITAR aggregates situational severity and cumulative extent into a unified score using a bounded recursive formulation, and is implemented as a lightweight BERT-family regressor that requires minimal runtime cost. The authors validate SITAR through extrinsic and intrinsic protocols, showing it tracks capability scaling and provides diagnostic utility across diverse datasets and models.
arXiv AI
Aug 13
Confidence Calibration of Deep Learning Systems
arXiv:2608. 12100v1 Announce Type: cross Abstract: In high-stakes applications, reliable confidence estimates are as important as the predictions themselves.
arXiv AI
Jun 9
GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model
arXiv:2512. 20978v2 Announce Type: replace-cross Abstract: Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech.
arXiv Computation and Language
Oct 9
Beyond Speech Captions: Speech-Rewarded Style Planning for Conversational Text-to-Speech
The paper introduces Speech-Rewarded Style Planning (SRSP), a method that trains a text-based style planner using a frozen downstream TTS model. SRSP generates candidate style instructions from dialogue history and response text, optimizing them with group-relative policy optimization (GRPO) where the reward is the teacher‑forced likelihood of target speech tokens. Experiments on an English subset of the ISCSLP 2026 CoT‑TTS corpus show that SRSP outperforms baseline LLM and target‑audio‑informed captioning approaches in speech‑style and emotion similarity, mel‑cepstral distortion, and contextual appropriateness. "whyItMatters":"The study demonstrates a more effective way to align natural‑language style descriptions with actual acoustic output, improving controllability and fidelity in conversational text‑to‑speech systems."