Emo-LiveSync is a real‑time universal lip‑sync video generation framework that builds on LivePortrait and a warp‑based portrait animation model. By aligning audio signals to an implicit motion space through knowledge distillation from a pre‑trained video‑to‑motion model, it achieves high visual fidelity and lip‑sync accuracy while remaining computationally efficient. The method also introduces an expressive control gate for explicit emotion manipulation, enabling content enrichment and generalization to stylized characters beyond photo‑realistic humans.
arXiv:2606. 28568v1 Announce Type: cross Abstract: Speech-driven 3D facial animation methods face significant challenges in simultaneously achieving high-fidelity motion and precise artistic control at production quality.
arXiv:2503. 14295v3 Announce Type: replace-cross Abstract: Recent advancements in audio-driven talking face generation have made great progress in lip synchronization.
The paper introduces an end‑to‑end framework for personalized audio‑driven facial motion that operates in real time without look‑ahead. It combines a causal multi‑resolution motion tokenizer, which captures both global temporal context and fine articulatory details, with a multi‑modal style retriever that pulls stylistic priors from arbitrary reference footage using ongoing audio and motion queries. This approach allows high‑fidelity, identity‑consistent animation from just a few casually recorded clips, outperforming existing methods in lip‑sync accuracy, identity consistency, and perceived realism while maintaining real‑time streaming constraints.
Multi-modal talking avatar synthesis aims to generate realistic talking videos from a reference portrait and speech. Despite rapid progress in diffusion-based methods, existing approaches still strugg...
arXiv:2608.00663v2 Announce Type: replace Abstract: Audio-driven emotional talking face generation aims to synthesize realistic videos with expressive facial dynamics. However, existing methods strug...
arXiv:2610.27947v1 Announce Type: new Abstract: Human-centric audio-visual editing requires precise alignment of lip movements, gestures, and expressions with speech. Existing methods process inputs...
arXiv:2609.22264v1 Announce Type: cross Abstract: State-of-the-art models for audio-driven digital human generation have achieved photo-realistic results in talking-head synthesis. However, extending...
FacEDiT is a mask‑controlled model for editing local segments of talking head videos by infilling facial motion from speech and surrounding context. It uses a self‑supervised pretext task that masks a motion span during training, enabling variable‑duration local rewriting while preserving identity, unedited regions, and boundary continuity. The authors also release FacEDiTBench, a benchmark covering diverse edit types and lengths, and demonstrate that FacEDiT achieves accurate, speech‑aligned edits with strong identity preservation and seamless transitions.
Recent advances in LipSync generation technology have led to the creation of highly realistic videos, posing severe societal risks. However, existing defense strategies struggle against LipSync forger...
arXiv:2609.18632v1 Announce Type: new Abstract: Multi-modal talking avatar synthesis aims to generate realistic talking videos from a reference portrait and speech. Despite rapid progress in diffusio...
Audio-driven talking head synthesis has achieved impressive progress in lip synchronization and visual quality, yet generating expressive emotional avatars with controllable intensity remains challenging, especially under real-time constraints. In this paper, we present GaussianEmoTalker, an audio-driven framework for real-time emotional talking head synthesis based on 3D Gaussian Splatting.
arXiv:2608. 15110v1 Announce Type: cross Abstract: Emotional 3D talking head generation aims to synthesize expressive facial animations with accurate lip synchronization.