arXiv Computer Vision
Head-Pose-Aware Visual Speech Recognition with FiLM Modulation
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Sep 25
Pose Adaptive Dynamic FiLM Modulation for Visual Speech Recognition
The paper introduces Pose Adaptive Dynamic FiLM Modulation for Visual Speech Recognition, addressing head‑pose variation that causes appearance changes in VSR. It proposes a Dynamic Residual FiLM (DR‑FiLM) modulator that predicts input‑dependent weights to control pose‑conditioned modulation strength. Experiments on LRS2 and LRS3 show that DR‑FiLM reduces phoneme error rates compared to unweighted multi‑pathway modulation and that deeper FiLM pathways receive higher weights as head‑pose variation increases.
arXiv Computer Vision
Oct 15
Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling
arXiv:2610.19559v1 Announce Type: new Abstract: Lipreading remains challenging due to the inherent ambiguity of visual speech cues and their sensitivity to head pose and video resolution. In this wor...
Hugging Face Trending Papers
Oct 14
Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling
Lipreading remains challenging due to the inherent ambiguity of visual speech cues and their sensitivity to head pose and video resolution. In this work, we propose FLARE, a framework for robust visua...
arXiv Computation and Language
Sep 21
Curriculum-Based Noise Adaptation for Phoneme-to-Text Reconstruction in Visual Speech Recognition
arXiv:2609.20839v1 Announce Type: new Abstract: Phoneme-centric visual speech recognition reconstructs sentences from intermediate phoneme predictions, making overall recognition performance highly d...
arXiv AI
Aug 7
PD-GS: Phoneme-Driven 3DGS for Audio-Driven Talking Heads
arXiv:2608. 05218v1 Announce Type: new Abstract: 3D Gaussian Splatting (3DGS) enables fast, photorealistic talking-head rendering, yet accurate lip articulation remains elusive: mouth motion is often over-smoothed and may violate hard articulatory constraints such as bilabial closures, producing the notorious ``leaky mouth'' artifact.
arXiv Computation and Language
Sep 30
Bootstrapping Audiovisual Speech Recognition in Zero-AV-Resource Scenarios
arXiv:2603.08249v2 Announce Type: replace-cross Abstract: Audiovisual speech recognition (AVSR) combines acoustic and visual cues to improve transcription robustness under challenging conditions but...