arXiv Computer Vision
Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
Hugging Face Trending Papers
6d ago
Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling
Lipreading remains challenging due to the inherent ambiguity of visual speech cues and their sensitivity to head pose and video resolution. In this work, we propose FLARE, a framework for robust visua...
arXiv Computer Vision
Sep 15
Head-Pose-Aware Visual Speech Recognition with FiLM Modulation
arXiv:2606.00751v2 Announce Type: replace Abstract: Visual Speech Recognition (VSR) aims to recognize speech from visual cues such as lip movements. Still, its performance is fundamentally limited by...
arXiv AI
Aug 7
PD-GS: Phoneme-Driven 3DGS for Audio-Driven Talking Heads
arXiv:2608. 05218v1 Announce Type: new Abstract: 3D Gaussian Splatting (3DGS) enables fast, photorealistic talking-head rendering, yet accurate lip articulation remains elusive: mouth motion is often over-smoothed and may violate hard articulatory constraints such as bilabial closures, producing the notorious ``leaky mouth'' artifact.
arXiv AI
Jul 31
Zero-Shot Face-to-Speech Synthesis via Latent Space Adaptation of a Style-Diffusion TTS Model
arXiv:2607. 26742v1 Announce Type: cross Abstract: Zero-shot text-to-speech (TTS) clones a voice from a short audio prompt, but this reliance on reference audio is a barrier when only visual information is available, e.
arXiv Computer Vision
1d ago
JAVIS: Joint Audio-Visual Interaction and Streaming for Real-Time Conversational Agents
arXiv:2610.22499v1 Announce Type: new Abstract: Human communication is inherently multimodal, conveying intent not only through linguistic and vocal cues but also through visual signals. However, exi...
arXiv Computer Vision
5d ago
Kandinsky S2V 1.0: Audio-Driven Avatar Video Generation
arXiv:2610.19290v1 Announce Type: new Abstract: We present Kandinsky S2V 1.0, an open family of speech-to-video models for controllable audio-driven human animation, comprising Kandinsky S2V 1.0 Lite...