arXiv AI By Alina Marcu, Dragos Costea, Cristina Lazar, Marius Leordeanu

Real-time body pose non-verbal communication with a consistency-based reliability measure

Read the original on arXiv AI →

arXiv:2606. 09390v1 Announce Type: cross Abstract: Body movement communicates intent at distances and in conditions where neither the face, nor speech can be captured.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Sep 7

Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

Motion-Omni is an end‑to‑end framework that jointly generates spoken dialogue and full‑body motion, producing speech, facial expressions, and hand, upper‑body, and lower‑body movements directly from the hidden states of a language model. The system requires joint training of the language model, speech generator, and motion generator to maintain audio‑motion alignment, and it is supervised using a scalable, model‑agnostic pipeline that pseudo‑labels 422,856 speech‑motion pairs. With a Qwen2.5‑7B‑Instruct backbone, Motion‑Omni‑Q7 achieves near‑cascade performance on motion metrics while being 5.4× faster, and it outperforms other non‑teacher cascades on beat correlation, diversity, and word error rate.

By Chengqian Ma, Wei Tao, Haoyu Zhang, Yiwen Guo