arXiv Computer Vision September 15, 2026 By Beibei Jing, Tianle Guo, Youjia Zhang, Zikai Song, Yawei Luo, Junqing Yu, Tao Guan, Wei Yang

MoVT: Video-Augmented Motion Tokenizer for Text-to-Motion Generation

Read the original on arXiv Computer Vision →

MoVT is a new framework for text‑to‑motion generation that uses a cross‑modal augmented motion tokenizer to project 3D motion tokens into 2D, enriching the motion codebook with real‑world video patterns. The enriched tokens are mapped back to 3D, creating aligned 3D and 2D codebooks that better capture intricate motions. These codebooks feed a generative masked transformer, which predicts masked motion tokens in a modality‑agnostic way, allowing text‑index pairs from the 2D codebook and annotated videos to further improve generation quality. Empirical tests show MoVT outperforms previous state‑of‑the‑art methods on several key metrics.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision

Aug 26

SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling

arXiv:2608.24334v1 Announce Type: new Abstract: Discrete motion representations have substantially advanced autoregressive text-to-motion generation. However, most motion tokenizers are optimized for...

By Tianlv Huang, Hetian Guo, Ziyi Cai, Song Wang, Yanping Zhang, Zipei Fan, Xuan Song, Guangming Wu, Xin Zheng

Hugging Face Trending Papers

Jun 28

Zero-Gated Language-conditioned Human Motion Prediction

Pose histories provide the core kinematic evidence for 3D human motion prediction, but they lack explicit high-level semantic guidance. This paper introduces ZGL, a lightweight language-conditioned predictor that uses captions of the observed motion as a semantic prior while preserving a strong motion backbone as the main source of dynamics.

llms multimodal benchmarks

arXiv AI

Sep 10

SeRV: Semantic-Aligned Residual Vector Quantization for American Sign Language Generation

arXiv:2609.05742v1 Announce Type: cross Abstract: American Sign Language (ASL) generation remains challenging due to limited paired text-ASL motion data and the difficulty of learning motion represen...

By Hongyu Wu, Xu Wu, Tianhao Wu, Jiawei Yu, Phuc Nguyen, Jian Liu, Yi Wu

llms nlp efficiency benchmarks safety

arXiv AI

Jun 2

Cross-Axis Feature Fusion with Joint-Wise Motion Difference Prediction for Text-Based 3D Human Motion Editing

arXiv:2606. 01014v1 Announce Type: cross Abstract: We address text-based 3D human motion editing, where the goal is to preserve the style and structure of a source motion while applying edits described in natural language.

By Gyojin Han, Junmo Kim

llms diffusion benchmarks safety

arXiv Computer Vision

4d ago

Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interaction

Uni-HOI is a unified framework that learns the joint distribution among text, human motion, and object motion for 4D human‑object interaction (HOI). It uses large language models and two motion‑specific VQ‑VAEs to convert heterogeneous motion data into token sequences, enabling seamless integration of all three modalities. A two‑stage training strategy first captures correlations on a large‑scale HOI dataset and then fine‑tunes for specific tasks, achieving strong performance on text‑driven HOI generation, object‑motion‑driven human motion generation, and human‑motion‑driven object motion prediction.

By Mengfei Zhang, Jinlu Zhang, Zhigang Tu

llms computer-vision efficiency

arXiv Machine Learning

Aug 19

From Diffusion to Flow: Efficient Motion Generation in MotionGPT3

The paper compares diffusion and rectified flow objectives within the MotionGPT3 framework for text-driven motion generation. Experiments on HumanML3D show that rectified flow converges faster, achieves strong test performance earlier, and matches or exceeds diffusion quality while requiring fewer sampling steps. The study isolates the generative objective’s impact, demonstrating that rectified flow’s benefits transfer to continuous-latent motion generation.

By Jaymin Bhan, JiHong Jeon, SangYeop Jeong

diffusion