arXiv:2609.26648v1 Announce Type: cross Abstract: Active speaker detection (ASD) requires reliable association between visible faces and acoustic speech, yet existing systems often degrade under chal...
arXiv:2505. 21954v2 Announce Type: replace-cross Abstract: We present UniTalk, a novel dataset emphasizing challenging scenarios to enhance model generalization for the task of active speaker detection (ASD).
The paper introduces a new target‑speaker unlearning task for automatic speech recognition (TSU‑ASR) that allows certain speakers to opt out of transcription while still indicating their presence. A lightweight Enrollment‑Conditioned Gating (ECG) module is added to a frozen dual‑stream speech LLM, enabling dynamic unlearning of new opt‑out speakers during inference. Experiments on AMI and AliMeeting datasets show significant drops in transcription accuracy for opt‑out speakers while preserving performance for retained speakers.
arXiv:2609.30631v1 Announce Type: cross Abstract: Online audio-visual target-speaker extraction aims to remove competing voices while preserving speech quality and bounding lookahead. Existing extrac...
arXiv:1707. 00835v1 Announce Type: cross Abstract: This contribution gives an overview of face recogni-tion algorithms, their implementation and practical uses.
The paper explores activation steering as a method to enhance automatic speech recognition (ASR) performance and demographic fairness. It compares two techniques—paired activation subtraction and supervised optimization—finding that supervised optimization effectively improves overall ASR accuracy and fairness across gender, age, and dialect. The study also demonstrates that steering vectors trained on acoustically similar speakers can yield competitive or superior results, enabling few‑shot or zero‑shot adaptation without requiring demographic labels.
arXiv:2609.10366v1 Announce Type: cross Abstract: While AVSR has achieved sub-1% word error rates on the standard LRS3 benchmark, its reliance on broadcast speech obscures whether this reflects true...
arXiv:2609.31193v1 Announce Type: new Abstract: Current Audio-Visual LLMs (AVLLMs) struggle with reasoning over videos featuring multi-speaker dialogues. In such videos, resolving "who says what" is...
The paper introduces a multi‑party backchannel prediction benchmark built from the AMI meeting corpus, featuring 682 masked‑listener views, 190 speakers, and 18,697 backchannel events. A state‑of‑the‑art dyadic model performs at chance when applied zero‑shot to meetings, but its frozen acoustic features are still informative, and retraining improves performance to an AUROC of 0.751. The study reveals that listener conditioning helps only for listeners seen during training, that speaker identity is entangled with useful cues, and that backchannel rates vary significantly across individuals, prompting the authors to report both AUROC and event‑F1 metrics. whyItMatters":"The benchmark and evaluation tools provide a standardized, person‑disjoint testbed for advancing multi‑party backchannel prediction research."
FASTDIAR is a frame‑level speaker encoder designed for real‑time streaming diarization on a single CPU thread. It converts a state‑of‑the‑art speaker recognition architecture into a causal model that emits an embedding every 80 ms from a two‑second window, and pairs it with online clustering that updates only when the stream’s self‑similarity changes. Trained via distillation from an utterance‑level teacher, it achieves the highest accuracy on low‑overlap benchmarks, degrades less with more speakers, and runs five times faster than real time.
CIViC (Conditional Input and Visual Interpretation Checks) is a four‑gate protocol designed to link interpretation to intervention, held‑out task comparison, or endpoint control in audio‑conditioned visual prediction. The authors evaluate CIViC on three proxy tasks—actor‑disjoint face‑raster tests, a synthetic audio‑feature mixture, and source‑video‑disjoint SpeakerVid—showing that different gates reveal distinct effects of audio manipulation on visual outputs. Across these tasks, CIViC demonstrates that output activation, held‑out error, and endpoint controls can support varied interpretations of audio influence on visual predictions.
arXiv:2606. 02724v1 Announce Type: cross Abstract: Audio-visual speaker tracking aims to localize and track active speakers by leveraging auditory and visual cues, enabling fine-grained, human-centric scene understanding.