← Back to all news
arXiv Computer Vision By Ekkasit Pinyoanuntapong, Ajinkya Deogade, Paul Streli, Wenjing Zhang, Joanna Materzynska, Pu Wang, Vittorio Ferrari, Jie Shen

InteractGesture: Progressive Chunk Guidance for Continuous Streaming Co-Speech Gesture Control

Read the original on arXiv Computer Vision →

InteractGesture is a model‑agnostic, inference‑time method that enables fine‑grained spatial control of individual joints in continuous streaming co‑speech gesture generation. It guides diffusion sampler latent estimates through a differentiable RVQ‑VAE decoder, backpropagating spatial control gradients to adjust motion latents during sampling. To address chunk‑wise dependency issues in streaming generation, the method introduces Progressive Chunk Guidance, a chunk‑window strategy that keeps an active set of editable chunk latents with staggered delays, allowing spatial constraints to propagate gradients backward across chunk boundaries and reducing boundary inconsistencies.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Aug 27

Super Star: Towards Streaming Real-time Interactive Agents for Digital Humans

The paper introduces a real‑time framework for generating co‑speech gestures for digital humans, coupling a streaming speech response module with a causal multimodal autoregressive gesture generator that uses only current speech and motion history. It also presents an offline data synthesis pipeline for virtual companion dialogues and a self‑evolving training loop that incorporates user feedback to continually adapt the model. Experiments show the system achieves a better latency‑quality trade‑off, stronger speech‑motion synchronization, and higher user preference than existing baselines.

By Wentao Jiang, Youchen Xie, Haidi Fan, Yajing Chen, Xin Wang, Ye Shi, Jingya Wang
agentsmultimodal
More like this →
arXiv Computer Vision
Sep 1

RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction

arXiv:2608.28693v1 Announce Type: cross Abstract: Enabling humanoid robots to respond to human speech with synchronized and semantically meaningful gestures is fundamental to natural human-robot inte...

By Zifan Wang, Ziang Ren, Pengyang Shi, Zirui Wang, Chenghuai Lin, Tianze Wang, Zekun Qi, Liangliang Zhao, He Wang, Li Yi
llmsdiffusionroboticsbenchmarkssafety
More like this →
arXiv Computer Vision
2d ago

Multi-Modal Controlled Coherent Motion Generation

The paper introduces MOCO, a diffusion-based framework that generates 3D avatar motions from concurrent multimodal inputs such as speech audio, text descriptions, and trajectory data. MOCO decouples motion generation by independently producing modality-specific motions at each denoising step and then assembling them according to spatial rules, iteratively refining the combined motion. This approach yields coherent, lifelike, and synchronized movements, outperforming existing baselines on a multimodal benchmark.

By Yifei Liu, Qiong Cao, Hongwei Yi, Huaiguang Jiang, Changxing Ding
diffusionmultimodalbenchmarks
More like this →