arXiv Computer Vision
Puppeteer: Object-Grounded Posture-Aware Co-Speech Gesture Generation
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
Hugging Face Trending Papers
Aug 31
Puppeteer: Object-Grounded Posture-Aware Co-Speech Gesture Generation
Generating co-speech gestures that are temporally coherent, semantically aligned with speech, and grounded with surrounding objects remains challenging. Prior speech-driven gesture models emphasize au...
arXiv Computer Vision
Aug 27
InteractGesture: Progressive Chunk Guidance for Continuous Streaming Co-Speech Gesture Control
InteractGesture is a model‑agnostic, inference‑time method that enables fine‑grained spatial control of individual joints in continuous streaming co‑speech gesture generation. It guides diffusion sampler latent estimates through a differentiable RVQ‑VAE decoder, backpropagating spatial control gradients to adjust motion latents during sampling. To address chunk‑wise dependency issues in streaming generation, the method introduces Progressive Chunk Guidance, a chunk‑window strategy that keeps an active set of editable chunk latents with staggered delays, allowing spatial constraints to propagate gradients backward across chunk boundaries and reducing boundary inconsistencies.
arXiv Computer Vision
Aug 27
Super Star: Towards Streaming Real-time Interactive Agents for Digital Humans
The paper introduces a real‑time framework for generating co‑speech gestures for digital humans, coupling a streaming speech response module with a causal multimodal autoregressive gesture generator that uses only current speech and motion history. It also presents an offline data synthesis pipeline for virtual companion dialogues and a self‑evolving training loop that incorporates user feedback to continually adapt the model. Experiments show the system achieves a better latency‑quality trade‑off, stronger speech‑motion synchronization, and higher user preference than existing baselines.
arXiv Computer Vision
Sep 1
RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction
arXiv:2608.28693v1 Announce Type: cross Abstract: Enabling humanoid robots to respond to human speech with synchronized and semantically meaningful gestures is fundamental to natural human-robot inte...
arXiv Computer Vision
Sep 3
DuoGesture: Motion-Grounded Semantic Conditioning and Biomechanical Beat Priors for Co-Speech Gesture Generation
DuoGesture is a co‑speech gesture generation model that separates gesture synthesis into a semantic stream and a beat stream, coordinated by a Semantic Variational Information Bottleneck that decides when semantic gestures override rhythmic motion. The semantic stream uses Motion‑Grounded Semantic Conditioning, replacing word embeddings with motion‑language representations to provide motion‑aligned semantic priors for rare gesture triggers. The beat stream is regularised by an Inertial Beat Prior, an anthropometry‑weighted arm‑chain module that reduces jitter and improves rhythmic consistency. Experiments show DuoGesture outperforms strong baselines and ablations confirm the complementary roles of semantic grounding, stochastic stream selection, and biomechanical regularisation.
arXiv Computer Vision
Aug 21
Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis
arXiv:2608. 20312v1 Announce Type: new Abstract: The capability to perceive and synthesize human-human interactions is fundamental to developing intelligent digital human systems.