The paper investigates what information is captured by egocentric video embeddings from models such as VideoMAEv2, CLIP, DINOv2, and VC-1, finding that the leading principal components largely reflect where the head‑mounted camera is looking. It also explores how to combine these video embeddings with body‑worn inertial sensor data for human activity recognition, showing that propagating inertial predictions over a video‑based graph can significantly improve classification performance (macro‑F1 from 0.648 to 0.852). The study demonstrates that video and inertial modalities play distinct roles—video identifies similar clips within a set, while inertial data labels the activities—yet fusing them directly in a single classifier or adding inertial edges to the graph does not yield better results.
arXiv:2608. 13283v1 Announce Type: new Abstract: Understanding motion in daily living requires context beyond kinematics, because similar inertial patterns during activities of daily living (ADLs) can reflect intentional stopping, object interaction, or pathological movement impairment.
Understanding motion in daily living requires context beyond kinematics, because similar inertial patterns during activities of daily living (ADLs) can reflect intentional stopping, object interaction, or pathological movement impairment. Egocentric vision provides task-related context that may help disambiguate these cases.
arXiv:2607. 13216v1 Announce Type: cross Abstract: Humans recognize movements effortlessly, even from noisy and complex visual input.
arXiv:2607. 15868v1 Announce Type: cross Abstract: Human motion capture from head-mounted devices (HMDs) offers a scalable way to acquire real-world human motion and interaction data, which is crucial for applications in embodied AI and VR/AR.
arXiv:2609.37938v1 Announce Type: cross Abstract: Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination. Yet aggr...
The paper introduces a redundancy-aware fusion framework for EgoExo proficiency estimation, which integrates fine-grained motion cues from egocentric views with spatial context from exocentric views. It identifies multiview redundancy and overfitting as key challenges and proposes two modules—AdaMVS for adaptive view selection and VIB-GB for compressing redundant signals—to address them. Experiments on EgoExo-4D and EgoExo-Fitness show that the method learns to select informative views and fuse them effectively, achieving state‑of‑the‑art results.
arXiv:2608. 15621v1 Announce Type: new Abstract: Human Activity Recognition (HAR) with self-administered wearables, such as at-home rehabilitation and exercise monitoring, often requires reattaching inertial measurement units (IMUs) across sessions.
MEgoVista is an offline pipeline that converts a single unprepared egocentric video into metric two‑hand and head motion within a gravity‑aligned world frame. It uniquely reconstructs motion in environments beyond studio volumes, uses calibrated stereo for absolute scale, and evaluates its outputs against independent optical capture to audit accuracy. The system thus expands the settings where high‑fidelity hand‑motion labels can be generated from natural, head‑worn recordings.
arXiv:2610.27622v1 Announce Type: cross Abstract: Handheld demonstration devices decouple data collection from the robot, but conventional designs burden the hand with a wrist camera, cables and a po...
EgoHRV is a method that estimates heart rate variability (HRV) and heart rate (HR) from the gaze cameras in egocentric headsets. It uses a 3D backbone and a low–high decomposition module to extract the blood volume pulse signal from gaze video, and aligns frequency‑domain representations of contact‑based and camera‑derived signals through cross‑domain pretraining. The approach achieves state‑of‑the‑art accuracy for HR and HRV estimation and, when integrated into EgoExo4D’s proficiency estimator, improves accuracy by 17.8%.
arXiv:2609.01276v1 Announce Type: new Abstract: Complete 3D perception from egocentric video requires recovering the surrounding scene and the wearer's full-body motion in a shared metric frame. Exis...
Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination. Yet aggregate cross-video accuracy conflates failures of lo...