DuoGesture is a co‑speech gesture generation model that separates gesture synthesis into a semantic stream and a beat stream, coordinated by a Semantic Variational Information Bottleneck that decides when semantic gestures override rhythmic motion. The semantic stream uses Motion‑Grounded Semantic Conditioning, replacing word embeddings with motion‑language representations to provide motion‑aligned semantic priors for rare gesture triggers. The beat stream is regularised by an Inertial Beat Prior, an anthropometry‑weighted arm‑chain module that reduces jitter and improves rhythmic consistency. Experiments show DuoGesture outperforms strong baselines and ablations confirm the complementary roles of semantic grounding, stochastic stream selection, and biomechanical regularisation.
arXiv:2609.36685v1 Announce Type: new Abstract: Co-speech gesture generation aims to synthesize natural gestures that are both temporally synchronized with speech and semantically consistent with the...
The paper introduces a new benchmark for evaluating holistic, semantics-aware co‑speech gesture generation, combining 13 objective metrics and a novel Semantic Gesture Preservation (SGP) metric that measures how well ground‑truth semantic gestures are retained. A perceptual study with 101 participants rates generated gestures on five dimensions, revealing that SGP aligns with speech‑aware human judgments while other metrics do not. Composite metrics tailored to each dimension further improve alignment, especially for animation errors and content match, underscoring the need to validate objective metrics against human perception.
arXiv:2608.28693v1 Announce Type: cross Abstract: Enabling humanoid robots to respond to human speech with synchronized and semantically meaningful gestures is fundamental to natural human-robot inte...
arXiv:2609.00369v1 Announce Type: new Abstract: Generating co-speech gestures that are temporally coherent, semantically aligned with speech, and grounded with surrounding objects remains challenging...
InteractGesture is a model‑agnostic, inference‑time method that enables fine‑grained spatial control of individual joints in continuous streaming co‑speech gesture generation. It guides diffusion sampler latent estimates through a differentiable RVQ‑VAE decoder, backpropagating spatial control gradients to adjust motion latents during sampling. To address chunk‑wise dependency issues in streaming generation, the method introduces Progressive Chunk Guidance, a chunk‑window strategy that keeps an active set of editable chunk latents with staggered delays, allowing spatial constraints to propagate gradients backward across chunk boundaries and reducing boundary inconsistencies.
Generating co-speech gestures that are temporally coherent, semantically aligned with speech, and grounded with surrounding objects remains challenging. Prior speech-driven gesture models emphasize au...
GestureFAR is a flow‑autoregressive framework that generates natural co‑speech gestures from streaming speech while preserving causality and continuous motion expressiveness. It autoregresses over continuous motion latents using a transformer for audio‑motion context and a flow‑matching head to sample the next latent. A head‑only flow distillation strategy further reduces latency by collapsing multi‑step flow sampling into a single network evaluation, enabling real‑time token‑causal generation with improved quality‑latency trade‑off on the BEAT2 benchmark.
arXiv:2606. 19935v1 Announce Type: new Abstract: Humanoid robots require co-speech motions that are not only expressive and speech-aligned, but also physically executable under embodiment constraints.
The paper introduces a real‑time framework for generating co‑speech gestures for digital humans, coupling a streaming speech response module with a causal multimodal autoregressive gesture generator that uses only current speech and motion history. It also presents an offline data synthesis pipeline for virtual companion dialogues and a self‑evolving training loop that incorporates user feedback to continually adapt the model. Experiments show the system achieves a better latency‑quality trade‑off, stronger speech‑motion synchronization, and higher user preference than existing baselines.
arXiv:2606. 30266v1 Announce Type: cross Abstract: Motion-language agents must possess the bidirectional capability to both understand human movement (motion-to-text, M2T) and generate it from natural language (text-to-motion, T2M).
EMODY Flow is a lightweight flow‑matching framework that generates synchronized full‑body motion and facial expressions conditioned on speech and emotion. It attaches to a frozen Qwen‑3 Omni model, reusing its audio codecs to drive two parallel DiT generators for SMPL‑X body pose and FLAME facial expressions. An auxiliary emotion classifier at training time restores emotion sensitivity, enabling EMODY Flow to achieve state‑of‑the‑art gesture quality on BEAT2 and zero‑shot facial animation on TFHP, with significant improvements in FGD, Beat Correlation, and Diversity metrics.
MyoSem is an EMG–action semantic alignment framework that maps hand electromyography signals and multi‑view natural‑language action descriptions into a shared feature space for bidirectional retrieval. It constructs semantic prototypes from text, encodes EMG with local activation‑aware representations, and aggregates evidence using semantic queries. Evaluated on multiple datasets and protocols, MyoSem outperforms existing methods in bidirectional EMG‑text retrieval, demonstrating its effectiveness in moving EMG hand action understanding toward language‑queryable semantic retrieval.