arXiv Machine Learning By Hao Yin, Paritosh Parmar, Lijun Gu, Lin Xu, Tianxiao Guo, Xiujin Liu, Tianyou Zheng, Yang Zhang, Weiwei Fu

MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching

Read the original on arXiv Machine Learning →

MyoMechanix is a multimodal dataset and framework for action quality assessment that incorporates muscle activity and other physiological signals alongside visual data. It contains over 7,500 samples of 20 weight‑loaded actions from 38 subjects, with synchronized RGB video, 3D pose, sEMG, and additional signals. The accompanying Fitness Knowledge Graph structures expert annotations into relationships among actions, phases, key steps, errors, and corrective feedback, enabling compositional scoring and interpretable assessment through the CUBIST engine. The project also introduces MyoMechanix‑AQA, MyoMechanix‑VideoQA, and a novel MyoMechanix‑Video2EMG task, demonstrating that multimodal sensing and structured representations improve performance, interpretability, and error attribution.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computer Vision
Oct 21

RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation

The paper introduces RoboJudge, a multimodal language model trained to evaluate embodied video generation. It presents RoboJudge-Bench, a benchmark of 800 human‑annotated videos assessing Physical Adherence and Instruction Alignment, and RoboJudge-Train, a 12K‑video dataset with human‑verified scores and rationales. Fine‑tuned on this data, RoboJudge‑9B achieves the highest agreement with human judgments, outperforming existing models and improving performance on external physics benchmarks.

By Siyuan Ma, Yu Qi, Cheng Ye, Ruotian Peng, Xinyi Xu, Zitiantao Lin, Zedong Cai, Yijian Huang, Weiyang Liu, Siyuan Wang, Yandong Wen