arXiv Computation and Language By Abhishek Gautam

Benchmarking Speaker Embedders in the Sub-Second Regime for Real-Time Voice AI Agents

Read the original on arXiv Computation and Language →

The paper evaluates several open‑source speaker embedding models—TitaNet‑Large, TitaNet‑Small, ECAPA‑TDNN, CAM++, and SpeakerNet—under a strict sub‑second latency constraint for real‑time voice AI agents. Using a controlled benchmark that injects a second speaker into real call recordings and a weighted human‑annotated production set, the authors find that (1) denoising before embedding generally worsens performance, (2) truncating audio below the full voice‑activity segment degrades all models, and (3) about half of the TitaNet embedding dimensions are redundant, encoding demographic attributes that add little discriminative value. They recommend running verification on raw, full voice‑activity audio, setting thresholds to favor precision, and using truncated embeddings to reduce similarity‑search cost.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI
Aug 18

Adding Voice Cloning to Text-to-Audio-Video Models with a Single Zero-Initialised Layer

arXiv:2608. 15690v1 Announce Type: cross Abstract: Text-to-audio-video (T2AV) generation models produce a video and its soundtrack from a textual description, but offer no control over whose voice speaks in the output.

By Ivan Mikheev, Viacheslav Vasilev, Anna Dmitrienko, Alexey Letunovskiy, Ivan Kirillov, Kirill Chernyshev, Denis Dimitrov
arXiv AI
Jul 17

RW-Voice-EQ Bench: A Real World Benchmark for Evaluating Voice AI Systems

arXiv:2607. 14846v1 Announce Type: cross Abstract: Current voice AI benchmarks typically evaluate isolated capabilities such as speech intelligibility, word error rate, or text-based dialogue quality, but they rarely test whether systems harness the acoustic information that distinguishes spoken language from its textual representation.

By David Ayllon, Alice Baird, Jeffrey Brooks, Franc Camps-Febrer, Jakub Piotr C{\l}apa, Theo Lebryk, Jens Madsen, Olya Ossipova, Sharath Rao, Hoon Shin, Tigran Soghbatyan, Georg Streich, Rashish Tandon, Panagiotis Tzirakis
arXiv Computation and Language
Sep 28

Inference-Time Target Speaker Unlearning in LLM-Based Automatic Speech Recognition

The paper introduces a new target‑speaker unlearning task for automatic speech recognition (TSU‑ASR) that allows certain speakers to opt out of transcription while still indicating their presence. A lightweight Enrollment‑Conditioned Gating (ECG) module is added to a frozen dual‑stream speech LLM, enabling dynamic unlearning of new opt‑out speakers during inference. Experiments on AMI and AliMeeting datasets show significant drops in transcription accuracy for opt‑out speakers while preserving performance for retained speakers.

By Bo Su, Yueru Yan, Thai Le