arXiv AI
Jun 8
dots.tts Technical Report
arXiv:2606. 07080v1 Announce Type: cross Abstract: We present dots.
SURE-Challenge: Evaluating Speech Evidence Before Speech-LLM Generation
The paper introduces the Speech-Unsupported Rejection Evaluation Challenge (SURE‑Challenge), a benchmark designed to test whether speech‑LLMs should accept or reject audio inputs before generating answers. Using LibriSpeech‑derived transcriptions paired with first‑word question answering, the authors evaluate various noise and silence conditions, and compare a simple energy‑plus‑Whisper‑score rule against a Qwen2‑Audio front‑end. On a 474‑row test set, the rule rejects 196 of 204 unsupported inputs while preserving accuracy on supported data, revealing a pre‑generation error mode that answer‑only scoring misses.
Related stories
arXiv AI
Jul 14
VoxENES 2026: Benchmarking Generalization of Speech Spoofing Detectors Against LLM-Era TTS and Voice Conversion
arXiv:2607. 11706v1 Announce Type: cross Abstract: Modern LLM-driven text-to-speech (TTS) and voice conversion (VC) systems produce synthetic speech that differs from the generators represented in many legacy spoofing benchmarks.
arXiv Computation and Language
Aug 31
Auditing Generative Audio Calls for Known-Task Audio-LLM Evaluation
The paper investigates how to evaluate audio‑language models by separating the use of acoustic evidence from the need to invoke a generative audio model. Using a controlled call‑decision framework, the authors compare policies that rely on transcript labels, encoder outputs from CLAP, AST, or WavLM, and optional calls to generative models such as Qwen2‑Audio, Qwen2.5‑Omni, or MOSS‑Audio. Results on the VocalSound dataset show that while transcript‑only accuracy is low (0.296), encoder‑only controls achieve high accuracy (≈0.85) without any generative calls, and adding generative calls yields only a marginal improvement (0.925 vs. 0.921).
arXiv Machine Learning
Aug 4
Domain-Specific Evaluation of Text-to-Speech Systems: A Multi-Metric Benchmarking Study
arXiv:2608. 02235v1 Announce Type: cross Abstract: Recent advances in neural text-to-speech (TTS) systems have substantially improved speech naturalness and intelligibility across many languages.
arXiv AI
3d ago
Where Does the Sound Go? Tracing Acoustic Information Loss in Audio-Conditioned LLMs
arXiv:2609.05871v1 Announce Type: cross Abstract: Audio-conditioned language models often underuse acoustic cues such as prosody, emotion, and non-speech sounds, raising the question of whether ASR-s...
arXiv AI
Aug 10
Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models
arXiv:2608. 06409v1 Announce Type: cross Abstract: Speech language models are increasingly evaluated on paralinguistic tasks by the accuracy of prompted answers, but answer accuracy combines failures at different stages of the audio-to-answer computation.
arXiv Computation and Language
Sep 1
Why We Need Speech to Evaluate Speech Translation
arXiv:2605.28227v2 Announce Type: replace Abstract: Speech translation models are increasingly capable of preserving speech-specific information (e.g., speaker gender, prosody, and emphasis), yet eva...
arXiv Machine Learning
Sep 4
VoxReason: Listener-Free Evaluation of Source-Grounded Speech Planning Before Synthesis
VoxReason introduces a listener‑free evaluation framework that measures whether a speech planning system’s delivery choices—such as pitch, energy, rate, pause, emphasis, and stance—are grounded in cited source records before any waveform is generated. The system outputs a source‑cited speaking plan and uses a deterministic verifier to check citation legality, slot agreement, unsupported states, schema validity, and counterfactual locality. Experiments on 1,440 source‑label cases show that simple slot accuracy can be misleading, while a 7B locality‑based repair model significantly improves plan‑slot accuracy and locality, and removing source records sharply reduces the grounded score. whyItMatters":"The framework provides a concrete, measurable way to ensure that expressive speech systems make source‑licensed planning decisions, addressing a source‑use failure that occurs before audio synthesis."
arXiv AI
Jun 18
Speaker Verification with Speech-Aware LLMs: Evaluation and Augmentation
arXiv:2603. 10827v2 Announce Type: replace-cross Abstract: Speech-aware large language models (LLMs) can accept speech inputs, yet their training objectives largely emphasize linguistic content or specific fields such as emotions or the speaker's gender, leaving it unclear whether they encode speaker identity.
arXiv Machine Learning
Jul 7
GRAFT: Grafted Reference Audio for Fine-grained Pronunciation in Zero-shot Text-to-Speech
arXiv:2607. 02633v1 Announce Type: new Abstract: We present GRAFT, a per-word pronunciation conditioning mechanism for text-to-speech neural codec language modeling.
arXiv AI
Aug 20
Aslema at NADI 2026: Augmentation through Fewshot for SLU
Aslema is a system developed for the NADI 2026 Shared Task 5, which involves intent recognition and slot filling. The team evaluated four omni LLMs in a zero‑shot setting and found that fine‑tuned models consistently outperform zero‑shot inference. They further improved performance by augmenting data with culturally grounded Tunisian Derja utterances generated by an LLM and synthetic speech produced via voice cloning, achieving top‑ranked results on the official test set.
arXiv Machine Learning
Aug 28
Interpretable, Fairly Evaluated Automated L2 Speaking Assessment that Beats the Single-Human Ceiling and Why Pause Encoding Does Not Change LLM Fluency Scores
The paper presents an interpretable, fair, and accurately benchmarked automated system for assessing second‑language English speaking. Using a hybrid of feature‑based speech‑timing metrics and a large language model (LLM) fluency judgment, the system achieves a Spearman correlation of 0.818 with the ICNALE Global Rating Archive, outperforming 81 % of trained human raters. A controlled study shows that encoding pauses into the LLM prompt does not meaningfully affect fluency scores, indicating that the system’s fluency signal derives from measurable speech‑timing features.