The paper introduces a new benchmark called Speech-Augmented Visually Grounded Contrastive Triplet Benchmark, comprising 10,150 images from 18 MENA countries, each paired with a supported statement and two plausible but unsupported alternatives. It defines contrastive instability as the rate at which multimodal models fail to resolve all statements within a triplet, distinguishing fragmented reasoning from complete failure. Experiments on recent multimodal models show that shifts in modality (text vs. speech) and language (English vs. Arabic) lead to significant triplet-level inconsistencies, especially when speech is used, which are not fully reflected by overall accuracy metrics.
By Basel Mousi, Fahim Dalvi, Shammur Chowdhury, Firoj Alam, Nadir Durrani
The paper introduces a cause-aware error recovery framework for cascaded Automatic Speech Recognition – Large Language Model (ASR‑LLM) pipelines in Spoken Dialogue Systems. It replaces simple ASR confidence filtering with precision‑focused detectors that use deep ASR latent representations to classify token‑level errors into perception, comprehension, and deletion failures. This fine‑grained diagnosis enables the LLM to execute targeted, multi‑turn clarification strategies, leading to a more than two‑fold increase in recall on domain‑shift errors and significant reductions in word error rate and downstream task errors across varied accents, distortions, and domains.
By Yizhou Peng, Ziyang Ma, Changsong Liu, Yi-Wen Chao, Xie Chen, Eng Siong Chng
arXiv:2610.24809v2 Announce Type: replace
Abstract: Speech recognition in real time in far-field conversational settings still incurs significant errors. Identifying potential error spans during stre...
By Sankalp Parashar, Preethi Jyothi, Sunita Sarawagi
The paper examines how different data‑augmentation techniques affect low‑resource automatic speech recognition beyond overall error rates. It compares speed perturbation, SpecAugment, and two text‑based audio synthesis methods across four low‑resource languages and English, analyzing phoneme‑level errors. The study finds that SpecAugment improves aggregate performance but increases tone and palatalization errors, while text‑based methods introduce consonant insertions; when combined, each method mitigates the other's weaknesses, yielding superadditive gains that diminish as data size grows to 100 hours.
By Katsumi Ibaraki, David Chiang
arXiv:2607. 23845v1 Announce Type: new Abstract: Other-initiated Self-repair, or in short Other-initiated Repair (OIR), is an essential mechanism in conversational interaction, whereby a recipient signals a problem in speaking, hearing, or understanding, prompting the previous speaker to resolve it.
By Anh Ngo, Nicolas Rollet, Catherine Pelachaud, Chlo\'e Clavel
The paper introduces a phoneme-guided text-to-speech (TTS) augmentation pipeline for automatic speech recognition (ASR) that links multilingual speech generation with candidate-text selection and reference-speech quality control. It proposes phoneme-frequency-guided selection (PFGS), which prioritizes candidate texts containing common phonetic content based on real ASR training transcripts. Experiments across four languages and 13 test sets show that random text selection improves recognition on 11 test sets, while PFGS further improves nine test sets with relative word error rate reductions up to 19.3%, and reference-speech filtering also contributes to performance gains.
By Zhen Wang, TianRui Wu, RongQi Han, Hao Wu, Wei Liang, Wei Xu
arXiv:2609.09889v1 Announce Type: new
Abstract: Automatic Speech Recognition (ASR) technology is fundamental to customer service automation and large-scale transcription. However, even advanced ASR m...
By Yonghyun Jun, Jimin Lee, Hwan Chang, Dongho Shin, Seolah Kim, Hwanhee Lee
arXiv:2606. 13464v1 Announce Type: cross Abstract: Automatic speech recognition (ASR) correction has traditionally focused on isolated utterances or short local contexts.
By Xinxin Li, Huiyao Chen, Meishan Zhang, Yunxin Li, Zulong Chen, Zhibo Ren, Xiaoqing Dong Baotian Hu, Min Zhang
The paper introduces ASCIL, a post‑ASR correction framework that re‑evaluates wake‑up intent by combining acoustic embeddings, linguistic cues, device context, and past misclassifications. ASCIL interprets both implicit (hesitation, disengagement, silence) and explicit (cancellation, repetition) signals as noisy indicators of misclassification, enabling online pattern updates without manual annotation. On a proprietary dataset of 3,667 interactions, ASCIL reduces errors by up to 54.27% relative on a session‑disjoint subset and 24.39% at a 0.90 threshold, while adding less than 60 ms of latency and improving intentional acceptance rates.
By Preeti Saraswat, Divya Neelagiri, Anil Yadav
The paper introduces a unified phoneme‑based TTS‑to‑ASR augmentation pipeline that uses a multilingual TTS model with language‑ID conditioning and incorporates grapheme‑to‑phoneme conversion, reference‑speech filtering, and candidate‑text selection. It proposes phoneme‑frequency‑guided selection (PFGS) to rank sentences based on phoneme frequencies from real ASR labels, and demonstrates that random augmentation and PFGS both improve ASR performance across Arabic, French, Italian, and Portuguese test sets, with PFGS yielding up to a 19.3% relative WER reduction. The study also shows that filtering reference speech can further lower WER by up to 0.59 points on certain datasets.
By Zhen Wang, TianRui Wu, RongQi Han, Hao Wu, Wei Liang
arXiv:2609.21663v1 Announce Type: new
Abstract: Word Error Rate (WER), the most commonly used metric for Automatic Speech Recognition (ASR), treats every lexical deviation from the reference as equal...
By Hritika Sharma, Thibault Ba\~neras-Roux, Alessandra Pinto, Petr Motlicek, Hyunggu Jung, Esa\'u Villatoro-Tello, Somang Nam
The paper introduces an LLM-assisted system for correcting speaker attribution errors during meetings. It combines streaming ASR, diarization, and concise LLM-generated summaries to guide users in providing brief corrective feedback, which updates the transcript and adds online speaker enrollments. The approach includes mechanisms to accurately interpret user corrections and a simulation for large-scale evaluation, achieving significant reductions in DER and speaker substitution error on the AMI headset test set, with a pilot usability study highlighting further improvements.
By Xinlu He, Yiwen Guan, Badrivishal Paurana, Pitipat Kongsomjit, Zilin Dai, Jacob Whitehill