arXiv Machine Learning

MorphCL: Morphological Contrastive Learning for Inertial-based Human Activity Recognition

MorphCL introduces a self‑supervised pretraining framework for inertial‑based human activity recognition that explicitly models global motion structure through morphology‑aware grouping. By leveraging motion primitives and domain‑specific descriptors, MorphCL improves linear probing and fine‑tuning performance of encoders by up to 15 percentage points in F1‑score. Compared with existing foundation models, MorphCL‑pretrained encoders achieve comparable or better linear probing results while using 4,600× less data, and the learned embeddings exhibit clear, morphologically meaningful clusters that better separate kinematically similar activities.

arXiv AI
Jul 9

Inertia-1: An Open Exploration of Wearable Motion Foundation Models

arXiv:2607. 06617v1 Announce Type: cross Abstract: Wearable motion sensing provides a continuous and scalable window into human behavior and health, making it a natural fit for foundation models, yet its pretraining and scaling principles remain poorly understood.

By Zongzhe Xu, Aakarsh Anand, Sarah Jiang, Chuntung Zhuang, Zitao Shuai, Sriram Sankararaman, Yuzhe Yang
arXiv Machine Learning
Aug 28

HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition

HALO is a heterogeneity‑aware, language‑aligned foundation model for inertial measurement unit (IMU) based human activity recognition. It uses a two‑stage training process: first, a self‑supervised encoder learns to handle diverse sensor configurations and natural‑language sensor descriptions; second, the encoder is aligned with text embeddings through synonym‑aware contrastive learning, enabling open‑set recognition via cosine similarity. Trained on ten public HAR datasets, HALO outperforms five state‑of‑the‑art baselines across eight metrics while using only ~35 M parameters, and improves zero‑shot open‑set accuracy by 13.7 percentage points over 87 training labels.

By Zihan Ding, Liyu Zhang, Xiaomin Ouyang
arXiv Machine Learning
Aug 6

Prototype-based Self-Supervised Multimodal Learning for PPG and Accelerometry Signals

arXiv:2510. 09764v2 Announce Type: replace Abstract: Modeling multi-modal time-series data is critical for capturing system-level dynamics, particularly in biosignals where modalities such as ECG, PPG, EDA, and accelerometry provide complementary perspectives on interconnected physiological processes.

By Wanting Mao, Maxwell A Xu, Harish Haresamudram, Mithun Saha, Santosh Kumar, James Matthew Rehg
arXiv AI
Jun 4

Gravity-Aware Hierarchical Routing for Lightweight SensorLLM on Human Activity Recognition

arXiv:2606. 04019v1 Announce Type: cross Abstract: Recent studies on sensor-language alignment have shown that two-stage frameworks can improve the semantic modeling ability of wearable-sensor human activity recognition (HAR), where SensorLLM-style methods first perform motion-to-language alignment and then fine-tune the model for downstream tasks.

By Hao Li, Mingrui Zheng, Yasuyuki Tahara, Yuichi Sei
arXiv Computer Vision
Oct 5

An Elastic Shape Variational Autoencoder for Skeleton Pose Trajectories

The paper introduces the Elastic Shape Variational Autoencoder (ES‑VAE), a geometry‑aware generative model for skeletal pose trajectories that uses the transported square‑root velocity field representation on Kendall's shape manifold to remove rigid transformations and temporal rate variability. ES‑VAE maps sequences to a low‑dimensional latent space via the Riemannian logarithm map and reconstructs them using the exponential map. Experiments on gait analysis for clinical mobility scoring and action recognition on the NTU RGB+D dataset show that ES‑VAE outperforms standard VAEs and several sequence‑modeling baselines.

By Arafat Rahman, Shashwat Kumar, Laura E. Barnes, Anuj Srivastava
arXiv Computer Vision
Sep 7

Object Concepts Emerge from Motion

The paper introduces a biologically inspired framework that learns object‑centric visual representations from raw videos without human annotations or camera calibration. By using motion boundaries detected via optical flow and clustering to create pseudo‑instance masks, the method supervises a single‑image encoder with pixel‑level pairwise metric learning. Training on 195 million pseudo‑labeled frames and expanding to 421 million frames through Motion‑Verified Self‑Training, the approach yields Swin‑based encoders that outperform or match supervised and self‑supervised baselines on tasks such as monocular depth estimation, 3D object detection, 3D occupancy prediction, and end‑to‑end planning.

By Boshi Li, Xiaohui Wang, Xiaoyang Wu, Zhichao Li, Ya Yang, Naiyan Wang