← Back to all news
arXiv Computer Vision By Yifei Liu, Qiong Cao, Hongwei Yi, Huaiguang Jiang, Changxing Ding

Multi-Modal Controlled Coherent Motion Generation

Read the original on arXiv Computer Vision →

The paper introduces MOCO, a diffusion-based framework that generates 3D avatar motions from concurrent multimodal inputs such as speech audio, text descriptions, and trajectory data. MOCO decouples motion generation by independently producing modality-specific motions at each denoising step and then assembling them according to spatial rules, iteratively refining the combined motion. This approach yields coherent, lifelike, and synchronized movements, outperforming existing baselines on a multimodal benchmark.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Aug 27

InteractGesture: Progressive Chunk Guidance for Continuous Streaming Co-Speech Gesture Control

InteractGesture is a model‑agnostic, inference‑time method that enables fine‑grained spatial control of individual joints in continuous streaming co‑speech gesture generation. It guides diffusion sampler latent estimates through a differentiable RVQ‑VAE decoder, backpropagating spatial control gradients to adjust motion latents during sampling. To address chunk‑wise dependency issues in streaming generation, the method introduces Progressive Chunk Guidance, a chunk‑window strategy that keeps an active set of editable chunk latents with staggered delays, allowing spatial constraints to propagate gradients backward across chunk boundaries and reducing boundary inconsistencies.

By Ekkasit Pinyoanuntapong, Ajinkya Deogade, Paul Streli, Wenjing Zhang, Joanna Materzynska, Pu Wang, Vittorio Ferrari, Jie Shen
diffusion
More like this →