arXiv Computer Vision

Pose2Muscle: Structured Spatio-Temporal Decoding for Discrete Muscle Activity Estimation from Human Pose

Pose2Muscle is a pose-driven framework that estimates discrete muscle activity states without requiring surface electromyography (sEMG) during inference. It reformulates muscle estimation as a structured prediction problem, using multi-scale spatio-temporal attention and a directed acyclic graph-based decoder to capture motion patterns and maintain multiple candidate hypotheses. The authors introduce the PoseEMG-43 dataset, comprising 2,992 movement instances from 43 daily-life actions performed by 14 participants, and demonstrate that Pose2Muscle outperforms baseline methods with high accuracy and correlation metrics.

arXiv Machine Learning
Aug 27

MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching

MyoMechanix is a multimodal dataset and framework for action quality assessment that incorporates muscle activity and other physiological signals alongside visual data. It contains over 7,500 samples of 20 weight‑loaded actions from 38 subjects, with synchronized RGB video, 3D pose, sEMG, and additional signals. The accompanying Fitness Knowledge Graph structures expert annotations into relationships among actions, phases, key steps, errors, and corrective feedback, enabling compositional scoring and interpretable assessment through the CUBIST engine. The project also introduces MyoMechanix‑AQA, MyoMechanix‑VideoQA, and a novel MyoMechanix‑Video2EMG task, demonstrating that multimodal sensing and structured representations improve performance, interpretability, and error attribution.

By Hao Yin, Paritosh Parmar, Lijun Gu, Lin Xu, Tianxiao Guo, Xiujin Liu, Tianyou Zheng, Yang Zhang, Weiwei Fu
arXiv Computer Vision
Sep 14

PhysioAI: Clinical Knowledge-Guided Semantic Supervision for Skeleton-Based Physiotherapy Action Recognition

PhysioAI introduces a clinical knowledge‑guided framework that injects structured physiotherapy knowledge into skeleton‑based action recognition models. By combining graph‑based spatiotemporal modeling with semantic anchors derived from a Clinical Knowledge Dictionary encoded via a frozen CLIP model, PhysioAI improves training of skeleton representations while requiring only skeleton inputs at inference. In subject‑disjoint evaluations, it outperforms existing methods on KiMoRe, Hard‑67, and UI‑PRMD benchmarks, achieving up to 99.03% accuracy on KiMoRe overall.

By Jie Cao, Euijoon Ahn, Anwar Hassan, Jinman Kim