arXiv Computer Vision
Oct 12

What Egocentric Video Embeddings Encode and How to Combine Them with Inertial Sensors for Human Activity Recognition

The paper investigates what information is captured by egocentric video embeddings from models such as VideoMAEv2, CLIP, DINOv2, and VC-1, finding that the leading principal components largely reflect where the head‑mounted camera is looking. It also explores how to combine these video embeddings with body‑worn inertial sensor data for human activity recognition, showing that propagating inertial predictions over a video‑based graph can significantly improve classification performance (macro‑F1 from 0.648 to 0.852). The study demonstrates that video and inertial modalities play distinct roles—video identifies similar clips within a set, while inertial data labels the activities—yet fusing them directly in a single classifier or adding inertial edges to the graph does not yield better results.

By Kansuke Habano
arXiv Machine Learning
Sep 23

GINIO: A Geometric SO(3)-Equivariant Interface for Neural Inertial Odometry

GINIO is a geometric SO(3)-equivariant interface designed for neural inertial odometry that ensures learned measurements transform consistently under any IMU mounting convention. It predicts motion measurements and uncertainties that obey vector and tensor transformation laws, and introduces Last-Frame Alignment to enable efficient sensor-frame learning equivalent to world-frame training. The interface is instantiated in several architectures—filter-connected NIO, AirIO-style recurrent aerial prediction, EqNIO-style full-SO(3) canonicalization, and ResNet-style temporal backbones—achieving significant accuracy and efficiency gains across multiple benchmarks.

By Chankyo Kim, Minghan Zhu, Tzu-Yuan Lin, Avantika Rattan, Maani Ghaffari
arXiv Computer Vision
Aug 28

Camera Calibration Using Inaccurate and Asynchronous Discrete GPS Trajectory from Drones

The paper tackles the problem of calibrating a stationary camera’s yaw, pitch, and roll using a drone’s GPS trajectory, which suffers from altitude bias, time offset, and discrete sampling. It formulates a parameter estimation problem that jointly estimates the GPS altitude bias, time offset, and camera orientation biases, and proposes a maximum likelihood estimator based on Iterated Least Squares to handle the asynchronous, discrete GPS data. Simulation results show the estimator achieves accuracy close to the Cramér–Rao Lower Bound, with a recommended drone trajectory yielding calibration errors within 14% of the measurement error standard deviation.

By R. Yang, Y. Bar-Shalom, H. A. J. Huang