arXiv Computer Vision
LiveSync: Iterative Self-Training for Lip-Sync Generation
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Oct 7
Ariadne's Thread of LipSync: Unraveling Forgeries via Inconsistency between Lip Motions and Head Poses
arXiv:2610.08417v1 Announce Type: new Abstract: Recent advances in LipSync generation technology have led to the creation of highly realistic videos, posing severe societal risks. However, existing d...
arXiv Computer Vision
Sep 30
Beyond Lip Sync: Reference-Grounded Oral Refinement for Audio-Driven Portrait Animation
arXiv:2609.38019v1 Announce Type: new Abstract: We present RGOR (Reference-Grounded Oral Refinement), an audio-driven lip-sync framework that renders the mouth of the specific person being dubbed rat...
Hugging Face Trending Papers
Aug 6
Vorch-Streamer: Extending Human Audio-Visual Generation to Real-Time Long-Form Streaming
Real-time long-form avatar audio--video generation requires causal, continuous synthesis while maintaining audiovisual synchronization and visual consistency. Adapting a pretrained bidirectional model to this setting presents two key dilemmas.
Hugging Face Trending Papers
Oct 6
Ariadne's Thread of LipSync: Unraveling Forgeries via Inconsistency between Lip Motions and Head Poses
Recent advances in LipSync generation technology have led to the creation of highly realistic videos, posing severe societal risks. However, existing defense strategies struggle against LipSync forger...
arXiv Computer Vision
Sep 23
Vorch-Human: Unified Multi-Task Human-Centric Generation via Long-Horizon Continuation
Vorch-Human is a unified framework for human‑centric audio‑visual generation that handles multiple tasks—animating a person from speech, jointly generating speech and video from a voice reference, and synthesizing a scene from paired appearance and voice references—using a single dual‑stream audio‑video diffusion transformer. The model incorporates clean condition‑audio and condition‑video tokens, per‑token task embeddings, temporal position types, condition masks, and a shared multimodal prompt encoder to express diverse inputs such as driving speech, timbre examples, first frames, and subject images. A two‑level data pipeline supplies the necessary supervision by extracting speech, appearance, and timbre annotations from clips and linking consistent identity and outfit references across videos, while a frozen‑prefix recurrence enables long‑form audio‑driven generation with reduced boundary discontinuity and identity drift.
arXiv Computer Vision
5d ago
Kandinsky S2V 1.0: Audio-Driven Avatar Video Generation
arXiv:2610.19290v1 Announce Type: new Abstract: We present Kandinsky S2V 1.0, an open family of speech-to-video models for controllable audio-driven human animation, comprising Kandinsky S2V 1.0 Lite...