The paper introduces SITAR, a situational distortion metric for evaluating automatic speech recognition (ASR) systems. SITAR aggregates situational severity and cumulative extent into a unified score using a bounded recursive formulation, and is implemented as a lightweight BERT-family regressor that requires minimal runtime cost. The authors validate SITAR through extrinsic and intrinsic protocols, showing it tracks capability scaling and provides diagnostic utility across diverse datasets and models.
By Mingzhe Lu, Jinhao Liu, Yunpeng Li, Hang Su, Lichun Fan, Jian Luan
arXiv:2609.13602v1 Announce Type: new
Abstract: Voice agents often need to collect names, addresses, identifiers, dates, and times exactly, yet end-to-end benchmarks obscure where capture fails. We i...
By Soham Ray, Victor Barres
FA-Bench is an open framework that standardizes forced alignment evaluation by fixing transcript normalization, data splits, phone mapping, and scoring procedures. It offers two tracks: one with reference transcripts and another with recognizer outputs, evaluated on clean and four degraded audio conditions using 21 open models and 9 commercial APIs. The benchmark scores every utterance boundary with a tolerance‑based F1 metric, revealing systematic timing biases such as Whisper’s 150 ms early alignment and commercial APIs’ 50 ms late alignment.
By Wei Chu, Yuanzhe Dong, Ke Tan, Dong Han, Yichao Zhou, Ruchao Fan, Bingshen Mu, Jingbei Li, Vishwas Shetty, Sarthak Bisht, Ziyue Qiu, Massa Baali, Rita Singh, Bhisha Raj
The paper introduces a token‑level extension of Omni‑Temporal Classification (OTC) for automatic speech recognition, allowing unsupported tokens to be bypassed while preserving supervision for the rest of the word. Across 19 languages and three corpora, this token‑level OTC consistently outperforms standard CTC, achieving the lowest mean word error rate on every dataset and a 9.45% average relative WER reduction. A predictive‑entropy‑indexed schedule replaces epoch‑based relaxation, reducing training‑length dependence while maintaining performance.
By Saurabh Kumar, Diptiman Mohanta, Prasanta Kumar Ghosh
The paper introduces a cause-aware error recovery framework for cascaded Automatic Speech Recognition – Large Language Model (ASR‑LLM) pipelines in Spoken Dialogue Systems. It replaces simple ASR confidence filtering with precision‑focused detectors that use deep ASR latent representations to classify token‑level errors into perception, comprehension, and deletion failures. This fine‑grained diagnosis enables the LLM to execute targeted, multi‑turn clarification strategies, leading to a more than two‑fold increase in recall on domain‑shift errors and significant reductions in word error rate and downstream task errors across varied accents, distortions, and domains.
By Yizhou Peng, Ziyang Ma, Changsong Liu, Yi-Wen Chao, Xie Chen, Eng Siong Chng
arXiv:2606. 19595v1 Announce Type: cross Abstract: Voice agents deployed in structured workflows (customer service, healthcare scheduling, account management) must handle frequent user interruptions while maintaining progress through multi-step procedures.
By Ahmad Salimi, Wentao Ma, Yuzhi Tang, Dongming Shen, Mu Li, Alex Smola
AVERT is a method for spoken dialogue state tracking that improves upon a per-turn text editor by incorporating an audio-conditioned verifier to score candidate slot values. It addresses three types of recoverable errors—inconsistent values across turns, omitted slots, and values unsupported by audio—using three specialized operators: vote, add, and swap, each limited to relevant slots. On the SpokenWOZ dataset, AVERT achieves a joint goal accuracy of 40.13, surpassing both a base speech-LLM (33.04) and a text editor (38.34) without retraining, and matching the performance of a larger end‑to‑end system that processes the full spoken history.
By Chunggi Lee, Hanspeter Pfister
arXiv:2609.38867v1 Announce Type: new
Abstract: Large language model (LLM) computer-use agents are typically evaluated with clean written instructions, despite speech being an increasingly popular in...
By Terumi Chiba, Guangzhi Sun, Zheqi Yuan, Chao Zhang
arXiv:2609.21663v1 Announce Type: new
Abstract: Word Error Rate (WER), the most commonly used metric for Automatic Speech Recognition (ASR), treats every lexical deviation from the reference as equal...
By Hritika Sharma, Thibault Ba\~neras-Roux, Alessandra Pinto, Petr Motlicek, Hyunggu Jung, Esa\'u Villatoro-Tello, Somang Nam
arXiv:2609.18533v1 Announce Type: new
Abstract: Automatic speech recognition (ASR) systems exhibit unequal error rates across speaker groups, motivating interventions on their internal representation...
By Nicolas Bourrel, Abderrahmane Issam, Gerasimos Spanakis
The paper investigates how to evaluate generative audio large language models (Audio‑LLMs) on known closed‑set tasks by separating the decision to call a generative model from the use of acoustic evidence. It introduces a controlled call‑decision framework where a policy can choose between a transcript label, encoder evidence from CLAP, AST, or WavLM, or a generative call to Qwen2‑Audio, Qwen2.5‑Omni, or MOSS‑Audio, and measures the impact of generative calls on accuracy. Results on the VocalSound dataset show that while transcript‑only accuracy is low (0.296), encoder‑based controls achieve high accuracy (≈0.85) without any generative calls, and adding generative calls yields only a marginal improvement (0.925 vs. 0.921).
By Mengzhe Geng
The study investigates how automatic speech recognition (ASR) transcript normalization—specifically the removal of filled pauses, punctuation, and colloquial expansions—affects dialogue act classification. By testing nine transcript conditions across six classifiers and three conversational corpora, the authors find that heavy normalization consistently lowers Macro‑F1 scores, especially when using a comprehensive filler lexicon. Punctuation removal and selective filler removal also degrade performance, indicating that the impact of normalization varies with preprocessing choices, feature extraction methods, and the conversational context.
By Lucky Onyekwelu-Udoka, Md Shafiqul Islam