arXiv AI By Jeevan Neupane, Janardan Bhetwal, Kshitiz Paudel, Anup Aryal, Bipin Bashyal

Nepali-English Code-Mixed Speech Recognition: Data Preparation and Adaptation of a Speech Foundation Model

Read the original on arXiv AI →

The paper presents a study on adapting the SraVaani speech recognition model to Nepali-English code‑mixed speech. Using a 172.8‑hour corpus across six Nepali domains, the authors fine‑tune the 449M‑parameter hybrid TDT‑CTC model, addressing tokenizer limitations for digits and symbols by generating Nepali and English readings and selecting plausible forms via zero‑shot ASR. Full fine‑tuning reduces the test word error rate from 48.6% to 17.1%, matching Whisper large‑v3‑turbo, while experiments on freezing the encoder and excluding numeric clips illustrate the impact of model components and data content.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Oct 23

Synthetic-to-Real Code-Switched ASR: A Spontaneous Nepali-English Podcast Benchmark

The paper introduces a new benchmark of 13,044 spontaneous Nepali‑English code‑switched podcast utterances, totaling 15.45 h, and evaluates a two‑stage synthetic‑to‑real adaptation framework for ASR. Stage 1 uses supervised fine‑tuning on 0.95 h of synthetic speech, reducing Mixed Error Rate from 23.40 % to 21.52 % and switch‑point error rate from 35.89 % to 28.77 %. Stage 2 applies Group Relative Policy Optimization with script and transcription rewards, yielding modest but statistically significant further improvements, particularly in English script confusion and switch‑point accuracy. The study also shows that an auxiliary script reward term offers no benefit beyond MER‑only optimization in this setting, and that the synthetic SFT provides the main cross‑domain transfer to spontaneous human speech.

By Firoj Paudel
arXiv Computation and Language
Aug 24

Building and Evaluating a Synthetic Bengali Speech Resource for Telecom Customer Care

The paper introduces a synthetic Bengali speech dataset tailored for telecom customer‑care applications, comprising 10,000 audio‑text pairs (≈26.82 hours) with predefined train, validation, and test splits. The data were generated using OmniVoice voice‑cloning, and include both original and normalized transcripts for ASR/STT use. Automatic intelligibility evaluation with a fine‑tuned Whisper model shows an average WER of 2.54% and CER of 0.59%, indicating strong text‑audio consistency, while the authors note limitations of synthetic speech and STT‑based evaluation.

By Kawshik Kumar Paul, Md. Nafiul Alam Fuji