arXiv Computer Vision
Head-Pose-Aware Visual Speech Recognition with FiLM Modulation
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Aug 7
PD-GS: Phoneme-Driven 3DGS for Audio-Driven Talking Heads
arXiv:2608. 05218v1 Announce Type: new Abstract: 3D Gaussian Splatting (3DGS) enables fast, photorealistic talking-head rendering, yet accurate lip articulation remains elusive: mouth motion is often over-smoothed and may violate hard articulatory constraints such as bilabial closures, producing the notorious ``leaky mouth'' artifact.
arXiv Computer Vision
Sep 10
Candor-LR: A Dyadic Conversational Dataset for Audio-Visual Speech Recognition
arXiv:2609.10394v1 Announce Type: cross Abstract: Current audio-visual speech recognition (AVSR) benchmarks, like LRS3, rely heavily on clean, scripted and rehearsed speech. They fail to reflect the...
arXiv Computer Vision
Sep 3
From Visual Cues to Spoken Narration: Rethinking Audio Description
The paper introduces Cue2Narrate, a two‑stage pipeline that jointly predicts what visual events to narrate and when to insert the narration in long, untrimmed movie clips. It uses a dual‑head audio‑visual localizer to identify visual cue and narration windows, followed by a LoRA‑adapted vision‑language model that generates concise audio descriptions, trained with a Description Ranking Loss. The authors also present the LongLSMDC benchmark, comprising up to 8‑minute clips, and show that Cue2Narrate outperforms video‑only and audio‑only baselines by 5–12 points in average mAP and improves AD generation over fine‑tuned base VLMs.
arXiv Computation and Language
4d ago
Exploring Multimodal Turn-Taking Cues in Face-to-Face Conversation using Voice Activity Projection
arXiv:2609.14666v1 Announce Type: cross Abstract: Turn-taking is a fundamental component of spoken interaction, and while humans naturally rely on both verbal and non-verbal signals, dialogue systems...
Hugging Face Trending Papers
Sep 1
From Visual Cues to Spoken Narration: Rethinking Audio Description
Audio Description (AD) provides spoken narration of visual events during dialogue gaps, making movies accessible to visually impaired audiences. The problem requires determining both what (which visua...
arXiv Computer Vision
Sep 10
AVSRBench: A Multi-Condition AVSR Benchmark
arXiv:2609.10366v1 Announce Type: cross Abstract: While AVSR has achieved sub-1% word error rates on the standard LRS3 benchmark, its reliance on broadcast speech obscures whether this reflects true...