The paper introduces two new corpus‑level, reference‑free metrics—Phoneme‑Cluster Mutual Information (PCMI) and Word Acoustic Consistency Score (WACS)—that use self‑supervised speech representations to evaluate forced alignment quality. PCMI quantifies how well aligned phoneme labels agree with clusters derived from SSL representations, while WACS assesses consistency across repeated word realizations via dynamic time warping of word representation sequences. Experiments on 85 languages from FLEURS and 45 languages in DoReCo show that both metrics degrade predictably under alignment perturbations, effectively distinguish high‑ from low‑quality alignments, and correlate strongly with traditional timestamp‑based measures, enabling scalable, multilingual alignment evaluation without manual annotations.
arXiv:2607. 21332v1 Announce Type: cross Abstract: Phonetic forced alignment is a key technique in phonetic research, yet existing alignment systems lack specialized models for low-resource language varieties.
Phonetic forced alignment is a key technique in phonetic research, yet existing alignment systems lack specialized models for low-resource language varieties. We address this by training text-dependent and text-independent aligners for Chengdu Mandarin using a 17-hour corpus and a custom G2P dictionary.
arXiv:2510.22172v2 Announce Type: replace-cross Abstract: The Continuous Integrate-and-Fire (CIF) mechanism provides effective alignment for non-autoregressive (NAR) speech recognition. This mechanis...
FA-Bench is an open framework that standardizes forced alignment evaluation by fixing transcript normalization, data splits, phone mapping, and scoring procedures. It offers two tracks: one with reference transcripts and another with recognizer outputs, evaluated on clean and four degraded audio conditions using 21 open models and 9 commercial APIs. The benchmark scores every utterance boundary with a tolerance‑based F1 metric, revealing systematic timing biases such as Whisper’s 150 ms early alignment and commercial APIs’ 50 ms late alignment.
The paper introduces Align Then Reason (ATR), a multilingual lip‑sync judge that aligns frame‑level lip representations with phonetic units of a candidate text line and then uses a language model to evaluate both content and timing. ATR achieves significant improvements over existing baselines on a seven‑language benchmark, with mean AUC gains of up to 59.4% for 2B reasoners and similar gains across other LLM families. The method also transfers well to unseen languages and outperforms lip‑reading baselines on real dubbing tasks such as dub‑line reranking and script‑to‑clip assignment.
arXiv:2609.23191v1 Announce Type: new Abstract: Speech-text space alignment is a multimodal representation learning method consisting to map different speech and text into a shared representation spa...
arXiv:2607. 06831v1 Announce Type: cross Abstract: Speech-to-text alignment means finding the temporal boundaries of each word in the audio.
arXiv:2610.22583v1 Announce Type: new Abstract: We show that forced aligners degrade on spontaneous, noisy real-world speech, and that fillers, laughter, and disfluencies corrupt word boundaries, imp...
arXiv:2609.09719v1 Announce Type: new Abstract: Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretr...
arXiv:2609.01246v1 Announce Type: new Abstract: Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet p...
arXiv:2507.02115v3 Announce Type: cross Abstract: Synthesizing second-language (L2) speech is potentially highly valued for L2 language learning experience and feedback. However, due to the lack of L...
The paper introduces SITAR, a situational distortion metric for evaluating automatic speech recognition (ASR) systems. SITAR aggregates situational severity and cumulative extent into a unified score using a bounded recursive formulation, and is implemented as a lightweight BERT-family regressor that requires minimal runtime cost. The authors validate SITAR through extrinsic and intrinsic protocols, showing it tracks capability scaling and provides diagnostic utility across diverse datasets and models.