arXiv Machine Learning

MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching

MyoMechanix is a multimodal dataset and framework for action quality assessment that incorporates muscle activity and other physiological signals alongside visual data. It contains over 7,500 samples of 20 weight‑loaded actions from 38 subjects, with synchronized RGB video, 3D pose, sEMG, and additional signals. The accompanying Fitness Knowledge Graph structures expert annotations into relationships among actions, phases, key steps, errors, and corrective feedback, enabling compositional scoring and interpretable assessment through the CUBIST engine. The project also introduces MyoMechanix‑AQA, MyoMechanix‑VideoQA, and a novel MyoMechanix‑Video2EMG task, demonstrating that multimodal sensing and structured representations improve performance, interpretability, and error attribution.

arXiv Computer Vision
4d ago

PhysioAI: Clinical Knowledge-Guided Semantic Supervision for Skeleton-Based Physiotherapy Action Recognition

PhysioAI introduces a clinical knowledge‑guided framework that injects structured physiotherapy knowledge into skeleton‑based action recognition models. By combining graph‑based spatiotemporal modeling with semantic anchors derived from a Clinical Knowledge Dictionary encoded via a frozen CLIP model, PhysioAI improves training of skeleton representations while requiring only skeleton inputs at inference. In subject‑disjoint evaluations, it outperforms existing methods on KiMoRe, Hard‑67, and UI‑PRMD benchmarks, achieving up to 99.03% accuracy on KiMoRe overall.

By Jie Cao, Euijoon Ahn, Anwar Hassan, Jinman Kim
arXiv AI
Jun 2

From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data

arXiv:2606. 00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models.

By Zhiyuan Feng, Qixiu Li, Huizhi Liang, Rushuai Yang, Yichao Shen, Zhiying Du, Zhaowei Zhang, Yu Deng, Li Zhao, Hao Zhao, Zongqing Lu, Oier Mees, Marc Pollefeys, Jiaolong Yang, Baining Guo
Hugging Face Trending Papers
Jul 13

From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence

Artificial general intelligence ultimately requires agents that can reason and act in the physical world. Action models, vision-language-action policies, and world models have advanced this goal, while World Action Models (WAMs) are particularly promising because they connect candidate interventions with predicted consequences.