The paper investigates how cluster assignments in soft targets influence speech representations in the S-JEPA model. By comparing original soft Gaussian mixture model (GMM) targets with counterfactual targets that keep the same probability values but alter the cluster assignments, the study finds that encoders trained with the original targets recover the soft distribution more accurately and provide better access to low-level acoustic and phonetic information. These results suggest that the specific cluster assignments in soft targets shape the learned representations beyond mere target matching.
arXiv:2609.37798v1 Announce Type: cross Abstract: Speech self-supervised learning aims to learn general-purpose representations for downstream speech tasks. However, current approaches rely on comple...
arXiv:2606. 28953v1 Announce Type: cross Abstract: Poisoning attacks entail attackers intentionally tampering with training data.
arXiv:2609.39162v1 Announce Type: cross Abstract: Speaker diarization systems based on speaker embeddings and neural diarization exploit complementary forms of speaker information, but their intermed...
arXiv:2607. 21820v1 Announce Type: cross Abstract: Audio deepfake detectors are trained to distinguish genuine speech from synthetic speech and often perform well on standard benchmarks.
The paper presents BiMamba2, a 47.88‑million‑parameter bidirectional Mamba‑2 encoder trained with masked discrete‑unit prediction for multilingual speech representation. It was trained on 250 hours of unlabeled speech from 67 languages and evaluated in the Unsupervised Speech in the Wild Challenge, achieving an Adjusted Rand Index of 0.735 for speaker clustering while reporting lower performance on language identification and character error rate compared to supervised baselines. The authors also discuss a discrepancy between local‑official metric scales and checkpoint rankings, underscoring the limits of in‑distribution diagnostics for predicting Dynabench probe outcomes.
The paper introduces asymmetric classifier‑free guidance (CFG) for target‑speaker ASR using Whisper, where a speaker‑conditioned branch predicts the target transcript and a speaker‑unconditioned branch predicts serialized multi‑speaker transcripts. CFG modulates the influence of speaker conditioning during decoding via a single guidance scale, which is first set globally on development data and then refined per utterance by a lightweight encoder‑based predictor while keeping the recognition model fixed. The resulting system yields up to 21.8% relative WER reduction over a condition‑only baseline and 5.6% over standard conditional decoding under domain shifts.
arXiv:2606. 07080v1 Announce Type: cross Abstract: We present dots.
arXiv:2512. 20978v2 Announce Type: replace-cross Abstract: Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech.
arXiv:2608.27783v3 Announce Type: replace-cross Abstract: Speech language models (speech LLMs) can generate plausible outputs from audio that contains no usable speech evidence. We study this failure...
Automatic speech recognition (ASR) systems exhibit unequal error rates across speaker groups, motivating interventions on their internal representations. We ask whether speaker-linked attributes that...
arXiv:2607. 03928v1 Announce Type: cross Abstract: Accent normalization (AN) seeks to convert non-native (L2) accented speech into standard (L1) speech while preserving speaker identity.
Large language models (LLMs) trained on next-token prediction exhibit remarkable in-context learning (ICL) abilities, yet the representations that support ICL remain poorly understood. We consider suc...