arXiv Machine Learning

MoCA: Multi-modal Cross-masked Autoencoder for Time Series in Digital Health

arXiv:2506. 02260v5 Announce Type: replace-cross Abstract: Wearable devices enable continuous multi-modal physiological and behavioral monitoring, yet analysis of these data streams faces fundamental challenges including the lack of gold-standard labels and incomplete sensor data.

arXiv Machine Learning

Aug 6

MoCA: Multi-modal Cross-masked Autoencoder for Digital Health Measurements

arXiv:2506. 02260v4 Announce Type: replace-cross Abstract: Wearable devices enable continuous multi-modal physiological and behavioral monitoring, yet analysis of these data streams faces fundamental challenges including the lack of gold-standard labels and incomplete sensor data.

By Howon Ryu, Yuliang Chen, Yacun Wang, Andrea Z. LaCroix, Chongzhi Di, Loki Natarajan, Yu Wang, Jingjing Zou

llms multimodal benchmarks

arXiv AI

Aug 10

Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations

arXiv:2608. 07385v1 Announce Type: cross Abstract: Learning disentangled representations is a key requirement for developing versatile, general-purpose, and sustainable models in multi-modal wearable computing.

By Ioannis Ziogas, Ensieh Khazaei, Bilal Taha, Aamna Al Shehhi, Ahsan H. Khandoker, Leontios J. Hadjileontiadis, Dimitrios Hatzinakos

llms multimodal

arXiv Machine Learning

Jul 31

HealthCAT: An Interpretable Encoder-only Transformer Framework for Health Indicator Prediction and Temporal Interpretation of Wearable Sensor Data

arXiv:2607. 27635v1 Announce Type: cross Abstract: Wearable sensors continuously capture fine-grained multivariate time-series data, providing opportunities to model behavioural patterns associated with health outcomes.

By Xiaotong Yu, Joshua Y. Kim, HaeJin Lee, Kalina Yacef

llms safety

arXiv Machine Learning

Jun 29

A Comparison of Fusion Techniques for Multi-Modal Human Activity Recognition on the HARMES Dataset

arXiv:2606. 27886v1 Announce Type: new Abstract: Recent advances in Human Activity Recognition (HAR) from wearable sensors have shown that multi-modal deep learning models consistently outperform their uni-modal counterparts.

By Ahmed Mohamady, Robin Burchard, Kristof Van Laerhoven

multimodal benchmarks

arXiv Machine Learning

Aug 6

Prototype-based Self-Supervised Multimodal Learning for PPG and Accelerometry Signals

arXiv:2510. 09764v2 Announce Type: replace Abstract: Modeling multi-modal time-series data is critical for capturing system-level dynamics, particularly in biosignals where modalities such as ECG, PPG, EDA, and accelerometry provide complementary perspectives on interconnected physiological processes.

By Wanting Mao, Maxwell A Xu, Harish Haresamudram, Mithun Saha, Santosh Kumar, James Matthew Rehg

rag multimodal

arXiv AI

Jun 9

Next-Token Prediction Learns Generalisable Representations of Sleep Physiology

arXiv:2606. 09605v1 Announce Type: new Abstract: Foundation models offer a promising route to compress multi-modal physiological signals into compact representations of human health, with broad applications across sleep medicine, cardiology, neurology and other healthcare domains.

By Jonathan F. Carter, Lionel Tarassenko

llms rag efficiency multimodal benchmarks

arXiv AI

Jun 2

Towards a General Intelligence and Interface for Wearable Health Data

arXiv:2605. 22759v2 Announce Type: replace Abstract: While ubiquitous wearable sensors capture a wealth of behavioral and physiological information, effectively transforming these signals into personalized health insights is challenging.

By Girish Narayanswamy, Maxwell A. Xu, A. Ali Heydari, Samy Abdel-Ghaffar, Marius Guerard, Kara Vaillancourt, Zhihan Zhang, Jake Garrison, Levi Albuquerque, Dimitris Spathis, Hong Yu, Hamid Palangi, Xuhai "Orson" Xu, David G. T. Barrett, Joseph Breda, Jed McGiffin, Yubin Kim, Yuwei Zhang, Naghmeh Rezaei, Samuel Solomon, Karan Ahuja, Tim Althoff, Jake Sunshine, Ming-Zher Poh, Benjamin Yetton, Ari Winbush, Nicholas B. Allen, James M. Rehg, Isaac Galatzer-Levy, Yun Liu, John Hernandez, Anupam Pathak, Conor Heneghan, Yuzhe Yang, Ahmed A. Metwally, Pushmeet Kohli, Mark Malhotra, Shwetak Patel, Xin Liu, Daniel McDuff

llms rag agents

arXiv AI

Aug 7

Is Self-Pretraining really useful to improve diagnosis in medical Time Series?

arXiv:2608. 06122v1 Announce Type: cross Abstract: Inspired by recent evidence that transformer architectures benefit from Self-PreTraining (SPT) on long-context benchmarks, we investigate whether similar gains extend to multimodal, multivariate, and even simple univariate medical time series.

By Omar Coser, Antonio Orvieto, Paolo Soda, Loredana Zollo

llms robotics multimodal benchmarks

arXiv Machine Learning

Jul 7

Modular Foundation Models for Time-Series Perception in Digital Twins

arXiv:2607. 03585v1 Announce Type: new Abstract: Engineering Digital Twins and Prognostics and Health Management (PHM) systems rely on robust perception modules to extract actionable information from heterogeneous and non-stationary time-series data.

By Quang Hung Pham, Ryad Zemouri, Martin Gagnon, Luc Vouligny

llms diffusion benchmarks safety

arXiv Machine Learning

Jun 5

PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Data

arXiv:2606. 06328v1 Announce Type: new Abstract: In healthcare, multimodal time series tasks often operate on incomplete observations in practice, for example when ECG segments are lost because electrodes detach or an entire respiratory channel is unavailable during overnight monitoring.

By Ziwen Kan, Wugeng Zheng, Tianlong Chen, Song Wang

rag diffusion multimodal benchmarks

arXiv Machine Learning

Jun 2

Longitudinal Multimodal Sensing of Physical Activity and Well-Being in Older Adults

arXiv:2606. 00345v1 Announce Type: new Abstract: Wearable and mobile sensing technologies enable continuous monitoring of human behavior and health in real-world settings.

By Flavio Di Martino, Mattia G. Campana, Marcello Magno, Lorenza Pratali, Franca Delmastro

multimodal safety

arXiv Machine Learning

Aug 11

Deep Multimodal Wearable Sensor Fusion for Detection of Body-Focused Repetitive Behaviors

arXiv:2608. 09830v1 Announce Type: new Abstract: Body-focused repetitive behaviors, such as hair pulling and skin picking, are compulsive motor actions commonly associated with obsessive-compulsive and anxiety disorders.

By Samaneh Rezaeimanesh, Mohsen Behradfar, Mohammad Fili, Guiping Hu

multimodal safety

Related stories