arXiv AI

Building a Production Greek-English Speech Recognizer

The paper details a multi‑month engineering effort to build Sophea, a production Greek‑English automatic speech recognition system. It evaluates the system against nine production gates—including word error rate, language identification, and hallucinations—across twenty‑three training iterations and two model architectures, finding that no single training‑data composition satisfies all gates. The authors describe a six‑stage data pipeline that dramatically reduces discarded Greek audio, isolate a hallucination defect to a specific training package, and demonstrate that a three‑model ROVER ensemble achieves full gate coverage and a 29 % relative WER improvement. A separate learned arbiter model is released on the Open ASR Leaderboard with 4.26 % average WER on English tests and 25.88 % on live Greek noisy traffic.

arXiv Computation and Language
Sep 24

Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts

The paper introduces a trilingual spoken hallucination detection benchmark covering English, Russian, and Kazakh news, with 12,013 samples that include synthetic alterations and severity levels, as well as 290 fact‑checked misinformation items. Detectors are evaluated in a reference‑free setting on text, ASR transcripts, and audio, revealing that most models underperform baseline classifiers, except Gemma‑3n on transcripts. Synthetic‑trained detectors achieve high macro‑F1 scores on real‑world misinformation, but Russian provenance analysis highlights model‑dependent signals that confound synthetic benchmarks.

By Meruyert Aristombayeva, Jason S. Lucas, Chaewan Chun, Dongwon Lee
arXiv AI
Jun 9

Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER (13.8% cWER)

arXiv:2606. 07608v1 Announce Type: cross Abstract: We present a systematic study of fine-tuning OpenAI's Whisper large-v3 for Swiss German ASR, using 1,367 hours of broadcast speech paired with Standard German subtitles as weak supervision.

By Felix Akeret
arXiv Computation and Language
Oct 23

Synthetic-to-Real Code-Switched ASR: A Spontaneous Nepali-English Podcast Benchmark

The paper introduces a new benchmark of 13,044 spontaneous Nepali‑English code‑switched podcast utterances, totaling 15.45 h, and evaluates a two‑stage synthetic‑to‑real adaptation framework for ASR. Stage 1 uses supervised fine‑tuning on 0.95 h of synthetic speech, reducing Mixed Error Rate from 23.40 % to 21.52 % and switch‑point error rate from 35.89 % to 28.77 %. Stage 2 applies Group Relative Policy Optimization with script and transcription rewards, yielding modest but statistically significant further improvements, particularly in English script confusion and switch‑point accuracy. The study also shows that an auxiliary script reward term offers no benefit beyond MER‑only optimization in this setting, and that the synthetic SFT provides the main cross‑domain transfer to spontaneous human speech.

By Firoj Paudel
arXiv Computation and Language
Oct 12

Error Rate is Not Enough: Costs of Augmentation in Low-Resource ASR

The paper examines how different data‑augmentation techniques affect low‑resource automatic speech recognition beyond overall error rates. It compares speed perturbation, SpecAugment, and two text‑based audio synthesis methods across four low‑resource languages and English, analyzing phoneme‑level errors. The study finds that SpecAugment improves aggregate performance but increases tone and palatalization errors, while text‑based methods introduce consonant insertions; when combined, each method mitigates the other's weaknesses, yielding superadditive gains that diminish as data size grows to 100 hours.

By Katsumi Ibaraki, David Chiang
arXiv Machine Learning
Sep 21

Beyond WER: Entity and Disfluency Recall in Accented Conversational ASR

The paper introduces a three‑stage pipeline to improve accented conversational ASR for speakers from India, Indonesia, and Latin America. It uses heuristic SQL filters to curate entity‑rich training data, regional LoRA adapters fine‑tuned on Qwen2.5‑Omni‑3B to generate both verbatim and corrected transcripts, and a six‑category error taxonomy validated by an LLM judge. The approach raises entity recall to 80‑85% and filler recall to 76‑86%, while keeping WER low (6‑10%) and outperforming Whisper and a commercial ASR on entity recall.

By Fiza Husain, Ankit Pandey, Yash Singh
arXiv Computation and Language
Oct 15

CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognition

The paper introduces CAFE, a method that confines Centered Kernel Alignment (CKA) feature distillation to language‑routed adapters in compact multilingual ASR models. By restricting the CKA gradient to adapters that English bypasses, CAFE mitigates the trade‑off where CKA improves non‑English WER at the cost of English performance. On the Open ASR Leaderboard, CAFE reduces average WER from 8.55% to 8.02% and outperforms an adapter‑only capacity control, achieving competitive results across 48 test sets in 24 languages with 254M parameters.

By Tianchi Liu, Nicholas Moratelli, Khurram Azeem Hashmi, Mohammadreza Zolfaghari, Rishabh Jain, Louis Lu, Changdae Park, Amril Nazir
arXiv AI
Sep 4

Building and Evaluating Fixed-Voice Thai TTS from Synthetic Speech

The paper presents a method for creating a compact fixed‑voice Thai text‑to‑speech system by training a student model on synthetic speech generated from a large voice‑cloning teacher. By using only a short 15‑second voice reference and carefully filtering synthetic data, the authors build an 82‑million‑parameter model, Wayu‑Paxa‑TTS‑Edge, that runs on device without reference audio. The system achieves strong performance—68.2 % challenge‑set keyword accuracy, 91.4 % pause precision, and low character error rates—while outperforming its teacher and approaching the quality of a larger Gemini 3.1 model.

By Kunat Pipatanakul, Potsawee Manakul, Warit Sirichotedumrong, Sittipong Sripaisarnmongkol, Pakorn Nathong, Phatrasek Jirabovonvisut