MEgoVista is an offline pipeline that converts a single unprepared egocentric video into metric two‑hand and head motion within a gravity‑aligned world frame. It uniquely reconstructs motion in environments beyond studio volumes, uses calibrated stereo for absolute scale, and evaluates its outputs against independent optical capture to audit accuracy. The system thus expands the settings where high‑fidelity hand‑motion labels can be generated from natural, head‑worn recordings.
By Jiangong Xiao (Northwestern Polytechnical University), Zhihao Zhang (Xi'an Jiaotong University), Yifei Dong (Maniformer), Chao Ma (Maniformer), Zhouyi Jin (Maniformer), Zhiwen Hou (Maniformer), Li Liu (Maniformer), Weihuang Chen (Xi'an Jiaotong University), Hongbin Sun (Xi'an Jiaotong University), Maoqing Yao (Maniformer)
arXiv:2610.03710v1 Announce Type: cross
Abstract: Inspired by human vision, we introduce a framework using active gaze to enable fine-grained bimanual manipulation with only a single stereo camera. E...
By Kush Hari, Justin Kerr, Nidhya Shivakumar, Samarth Mahapatra, Carmelo Sferrazza, Jiahui Lei, Jitendra Malik, C. Karen Liu, Ken Goldberg, Angjoo Kanazawa
INSPECT is a system that learns how a robot should choose its camera view during assembly inspection by observing a smart‑glasses assistant that answers part queries and guides the user. It uses techniques such as Presence‑Invariant TwinSwap for object evidence calibration, claim‑indexed supervision to separate evidence needs from camera changes, and object‑centered calibration to adapt view preferences to robot poses. In experiments on gearbox assemblies and angle‑grinder recordings, INSPECT outperforms other non‑oracle policies, improving view utility and decision accuracy.
By Di Wen, Kailun Yang, Wenhao Guo, Yitian Shi, Junwei Zheng, Yufan Chen, Ruiping Liu, Jiale Wei, Rania Rayyes, Kunyu Peng
arXiv:2610.06970v1 Announce Type: new
Abstract: Egocentric bimanual hand pose estimation is important for virtual interaction, wearable control, and rehabilitation, but visual observations are often...
By JiaCheng Ge, SiYu Zhang, ShengJie Li, XinTong Yang
The study investigates how well a single consumer earbud IMU can estimate 3D body pose and whether adding foot IMUs improves accuracy. Using a multimodal capture pipeline with RGB‑D video, an AirPods head IMU, and Striv insole IMUs, the authors benchmark pose estimation across various motions and train recurrent models (IMUPoser and MobilePoser). Results show that a head IMU alone achieves 79.0 mm rigid‑MPJPE and 0.809 macro‑F1 for foot contact, while adding foot IMUs does not significantly improve pose and can even degrade performance due to insole orientation errors.
arXiv:2605.07943v2 Announce Type: replace-cross
Abstract: Active vision -- where a policy controls its own gaze during manipulation -- has emerged as a key capability for imitation learning, with mul...
By Giacomo Spigler
The paper investigates what information is captured by egocentric video embeddings from models such as VideoMAEv2, CLIP, DINOv2, and VC-1, finding that the leading principal components largely reflect where the head‑mounted camera is looking. It also explores how to combine these video embeddings with body‑worn inertial sensor data for human activity recognition, showing that propagating inertial predictions over a video‑based graph can significantly improve classification performance (macro‑F1 from 0.648 to 0.852). The study demonstrates that video and inertial modalities play distinct roles—video identifies similar clips within a set, while inertial data labels the activities—yet fusing them directly in a single classifier or adding inertial edges to the graph does not yield better results.
By Kansuke Habano
WARP-VLA is a Vision‑Language‑Action model designed to be robust to variations in wrist‑camera viewpoints. It uses a Mixture‑of‑Experts architecture where each expert learns a view‑specific feature transformation, and a router selects the appropriate expert based on implicit view cues, eliminating the need for explicit camera extrinsic parameters. Experiments on the LIBERO benchmark show that WARP‑VLA doubles the success rate of pi‑0.5 from 39.2% to 78.3% under wrist‑view perturbations, and real‑robot tests confirm that the simulation‑learned adaptation transfers to diverse deployment settings.
By Junmyeong Lee, Dongmin Shin, Min-Gyu Park, Wooseok Jeon, Inho Chang, Hae-Gon Jeon
arXiv:2607. 15868v1 Announce Type: cross Abstract: Human motion capture from head-mounted devices (HMDs) offers a scalable way to acquire real-world human motion and interaction data, which is crucial for applications in embodied AI and VR/AR.
By Jiaxi Jiang, Bharat Lal Bhatnagar, Nan Yang, Lingni Ma, Sebastian Starke, Robin Kips, Nadine Bertsch, Christian Holz, Federica Bogo
Learning humanoid loco-manipulation from human demonstrations rests on observing how locomotion, posture, and object interaction evolve together as tasks unfold. Scaling collection across diverse envi...
ESTHER is an end‑to‑end model for egocentric stereo hand reconstruction that integrates stereo geometry, temporal reasoning, and a suitable output representation for wearable cameras. It is trained on pseudo‑labels from a calibrated pipeline and introduces the ESTHER3D benchmark, which pairs a large in‑the‑wild training set with a motion‑capture test set of true metric ground truth. Experiments demonstrate state‑of‑the‑art accuracy, strong generalization, and robustness to missing views, dropped frames, and extreme lighting or motion blur, while preserving metric scale even when reduced to a single monocular view.
By Hongyu Ma, Hairong Qu, Shiqi Zhao, Yongsong Yang, Peng Yin
arXiv:2610.07217v1 Announce Type: cross
Abstract: Despite its promise for scaling robot learning, egocentric manipulation data is still scarce today. Collection at scale requires vertically integrati...
By Grounded Superintelligence, BitRobot