arXiv Machine Learning
Aug 27

MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching

MyoMechanix is a multimodal dataset and framework for action quality assessment that incorporates muscle activity and other physiological signals alongside visual data. It contains over 7,500 samples of 20 weight‑loaded actions from 38 subjects, with synchronized RGB video, 3D pose, sEMG, and additional signals. The accompanying Fitness Knowledge Graph structures expert annotations into relationships among actions, phases, key steps, errors, and corrective feedback, enabling compositional scoring and interpretable assessment through the CUBIST engine. The project also introduces MyoMechanix‑AQA, MyoMechanix‑VideoQA, and a novel MyoMechanix‑Video2EMG task, demonstrating that multimodal sensing and structured representations improve performance, interpretability, and error attribution.

By Hao Yin, Paritosh Parmar, Lijun Gu, Lin Xu, Tianxiao Guo, Xiujin Liu, Tianyou Zheng, Yang Zhang, Weiwei Fu
arXiv AI
Sep 25

DrGait: Biomechanically Grounded Visual Reasoning for Interpretable Clinical Gait Analysis

DrGait is a training‑free framework that transforms Vision‑Language Models into clinical planners for gait analysis. It separates semantic reasoning from geometric perception using a Triage‑Verification‑Synthesis workflow, where hypotheses are generated, verified with deterministic biomechanical tools, and refined in a closed‑loop. This approach reduces hallucinations and produces transparent, audit‑ready clinical reports with competitive diagnostic accuracy.

By Xiangyu Yin, Shiqi Wang, Abrar Alamri, Yasir Aljohani, Weichen Liu, Goeran Fiedler, Wei Gao
arXiv Computer Vision
Oct 1

EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos

arXiv:2609.39378v1 Announce Type: new Abstract: Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolvin...

By Shulin Tian, Junsu Kim, Shuai Liu, Hao Li, Yujiao Shen, Sihan Li, Zhe Yang, Yeongon Kim, Feiyu Li, Jialin Wu, Yichi Zhang, Wenhui Wang, Runmao Yao, Yuhao Dong, Zhaoxi Chen, Fangzhou Hong, Antonino Furnari, Jingkang Yang, Hongyuan Zhu, Ziwei Liu
arXiv Computer Vision
6d ago

PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop

PhysVista is a new benchmark that evaluates physical intelligence in Vision‑Language Models (VLMs) by integrating perception, reasoning, and plausibility assessment into a closed cognitive loop. It distinguishes between event‑level and scale‑level reasoning and tests models on both real‑world and AI‑generated videos to provide a holistic, fine‑grained analysis of physical understanding. Experiments show significant gaps in VLMs’ physical reasoning and plausibility assessment, underscoring the need for more principled, physically grounded multimodal designs.

By Xinge Peng, Yiting Lu, Tianwu Zhi, Wen Wen, Jianzhao Liu, Xin Li, Zhibo Chen
arXiv Computer Vision
Sep 14

PhysioAI: Clinical Knowledge-Guided Semantic Supervision for Skeleton-Based Physiotherapy Action Recognition

PhysioAI introduces a clinical knowledge‑guided framework that injects structured physiotherapy knowledge into skeleton‑based action recognition models. By combining graph‑based spatiotemporal modeling with semantic anchors derived from a Clinical Knowledge Dictionary encoded via a frozen CLIP model, PhysioAI improves training of skeleton representations while requiring only skeleton inputs at inference. In subject‑disjoint evaluations, it outperforms existing methods on KiMoRe, Hard‑67, and UI‑PRMD benchmarks, achieving up to 99.03% accuracy on KiMoRe overall.

By Jie Cao, Euijoon Ahn, Anwar Hassan, Jinman Kim