← Back to all news
arXiv Computer Vision By Sangjoon Yu, Seungeun Lee, JoonHeyok Shin, Ji-Su Kang

Emo-LiveSync: Real-Time Universal Lip Synchronization with Explicit Emotion Control

Read the original on arXiv Computer Vision →

Emo-LiveSync is a real‑time universal lip‑sync video generation framework that builds on LivePortrait and a warp‑based portrait animation model. By aligning audio signals to an implicit motion space through knowledge distillation from a pre‑trained video‑to‑motion model, it achieves high visual fidelity and lip‑sync accuracy while remaining computationally efficient. The method also introduces an expressive control gate for explicit emotion manipulation, enabling content enrichment and generalization to stylized characters beyond photo‑realistic humans.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Sep 21

Personalizing Causal Audio-Driven Facial Motion via Dynamic Multi-modal Retrieval

The paper introduces an end‑to‑end framework for personalized audio‑driven facial motion that operates in real time without look‑ahead. It combines a causal multi‑resolution motion tokenizer, which captures both global temporal context and fine articulatory details, with a multi‑modal style retriever that pulls stylistic priors from arbitrary reference footage using ongoing audio and motion queries. This approach allows high‑fidelity, identity‑consistent animation from just a few casually recorded clips, outperforming existing methods in lip‑sync accuracy, identity consistency, and perceived realism while maintaining real‑time streaming constraints.

By Xuangeng Chu, Yu Han, Wei Mao, Shih-En Wei
ragnlpmultimodalbenchmarks
More like this →