Fine-grained Human Motion Understanding with Language Models

Fine-grained Human Motion Understanding with Language Models introduces FiGMo, an LLM-based model that represents motion as timestamp‑grounded skeletal poses, enabling reasoning over action order and duration. The model uses a unified pose encoder for both 2D and 3D skeletons, optionally incorporating RGB context, and relies on newly constructed pose‑to‑motion supervision to address limited pose‑level grounding. Experiments demonstrate that timestamps improve temporal localization and preserve physical timing under subsampling, leading FiGMo to achieve state‑of‑the‑art results on motion question answering, dense motion captioning, and action recognition with detected poses.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

• llms
• nlp
• benchmarks