MoVT is a new framework for text‑to‑motion generation that uses a cross‑modal augmented motion tokenizer to project 3D motion tokens into 2D, enriching the motion codebook with real‑world video patterns. The enriched tokens are mapped back to 3D, creating aligned 3D and 2D codebooks that better capture intricate motions. These codebooks feed a generative masked transformer, which predicts masked motion tokens in a modality‑agnostic way, allowing text‑index pairs from the 2D codebook and annotated videos to further improve generation quality. Empirical tests show MoVT outperforms previous state‑of‑the‑art methods on several key metrics.
arXiv:2608.24334v1 Announce Type: new Abstract: Discrete motion representations have substantially advanced autoregressive text-to-motion generation. However, most motion tokenizers are optimized for...
arXiv:2609.37495v1 Announce Type: new Abstract: Human motion generation plays an important role in applications such as character animation, virtual environments, and embodied interaction. While exis...
Pose histories provide the core kinematic evidence for 3D human motion prediction, but they lack explicit high-level semantic guidance. This paper introduces ZGL, a lightweight language-conditioned predictor that uses captions of the observed motion as a semantic prior while preserving a strong motion backbone as the main source of dynamics.
arXiv:2609.05742v1 Announce Type: cross Abstract: American Sign Language (ASL) generation remains challenging due to limited paired text-ASL motion data and the difficulty of learning motion represen...
arXiv:2606. 01014v1 Announce Type: cross Abstract: We address text-based 3D human motion editing, where the goal is to preserve the style and structure of a source motion while applying edits described in natural language.
arXiv:2609.23817v1 Announce Type: new Abstract: We present VISTA, a two-stage framework for generating stylized 3D human motion by fusing structural content from text prompts with expressive style fr...
TKCAM is a new framework for generating controllable camera motion conditioned on text and sparse keyframes, using a 12‑dimensional kinematic representation and a two‑stage masked transformer with residual vector quantization. It allows users to provide free‑form text prompts and RGB observations at selected timestamps, enabling temporally localized visual guidance for coherent trajectory synthesis. The authors also introduce the RealEstate10K‑Cap dataset and a Universal CLaTr Evaluator, showing that TKCAM outperforms recent baselines on FID, text‑motion matching, and retrieval metrics.
Generating high-quality and controllable camera motion is essential for AI-assisted cinematography, video synthesis, and 3D scene understanding. We introduce TKCAM, a Text- and Keyframe-conditioned CA...
The paper introduces ELMA, a new episodic long‑term motion dataset designed to benchmark anaphoric compositions in text‑to‑motion generation. ELMA contains densely annotated 30‑second clips extracted from in‑the‑wild videos, preserving cross‑segment dependencies that current models struggle to handle. Fine‑tuning state‑of‑the‑art models on ELMA improves both global physical realism and semantic alignment, establishing a robust baseline for long‑term motion synthesis.
Uni-HOI is a unified framework that learns the joint distribution among text, human motion, and object motion for 4D human‑object interaction (HOI). It uses large language models and two motion‑specific VQ‑VAEs to convert heterogeneous motion data into token sequences, enabling seamless integration of all three modalities. A two‑stage training strategy first captures correlations on a large‑scale HOI dataset and then fine‑tunes for specific tasks, achieving strong performance on text‑driven HOI generation, object‑motion‑driven human motion generation, and human‑motion‑driven object motion prediction.
arXiv:2412.00112v5 Announce Type: replace Abstract: Generating natural and expressive human motions from textual descriptions is challenging due to the complexity of coordinating full-body dynamics a...
The paper introduces T2M Mamba, a text‑to‑motion generation model that addresses two key shortcomings of prior work: the independent treatment of motion periodicity and keyframe saliency, and sensitivity to paraphrased text. It achieves this by employing a Periodicity‑Saliency Aware Mamba module that estimates keyframe weights with Density Peaks Clustering and motion periodicity via FFT‑accelerated autocorrelation, and a Periodic Differential Cross‑modal Alignment Module (PDCAM) that improves alignment between textual and motion embeddings. Experiments on HumanML3D and KIT‑ML datasets demonstrate significant performance gains, achieving an FID of 0.068 and improvements across all other metrics.