TouchSight is a monocular egocentric vision framework that predicts dense full-hand contact forces without tactile sensors. It uses 500 hours of pressure-glove data and a 20-hour TwinTouch-20H dataset where generative models render gloved recordings as bare-hand videos, bridging the appearance gap. The system outperforms previous methods on OakInk2, generalizes to unseen natural bare-hand egocentric videos, and improves as glove supervision increases.
arXiv:2608.15060v2 Announce Type: replace
Abstract: Touch is fundamental to dexterous manipulation, yet most egocentric human data increasingly used for robot learning lacks tactile information. Dire...
By Wenkang Zhang, Chengbo Yuan, Zicheng Zhang, Zhengxue Cheng, Yang Gao
arXiv:2610.10288v1 Announce Type: new
Abstract: Large-scale egocentric human interaction data is becoming an important source of physical supervision for embodied learning, yet video alone leaves the...
By Dayou Li, Hao Wang, Qianqian Yang, Zihao Zhu, Haoquan Fang, Ziyao Zeng, Yan Han, Zihan Wang, Yan Wang, Baoru Huang, Dilin Wang, Kenji Shimada, Yiyue Luo, Manling Li, Teresa Lv, Mustafa Mukadam, Rakesh Ranjan, Ruohan Zhang, Qi He, Changliu Liu, Xu Chen, Marco Pavone, Bangya Liu, Jiachen Li, Masayoshi Tomizuka, Zhiwen Fan
arXiv:2606. 09243v1 Announce Type: cross Abstract: Estimating full-hand grasp pressure from egocentric video is critical for immersive VR and robotic manipulation, yet dense tactile sensing often relies on intrusive hardware.
By Yuan Zeng, Yujia Shi, Tiao Tan, Xingting Li, Yaqi Qin, Zongqing Lu, Wenming Yang, Jing-Hao Xue, Qingmin Liao
Touch-JEPA is a framework that learns tactile representations from hand‑object videos using limited tactile supervision. It first trains a self‑supervised model that predicts masked visual features at hand‑surface locations, capturing latent contact dynamics. The model is then jointly fine‑tuned on a small labeled subset, improving contact discrimination and tactile magnitude estimation on human‑hand benchmarks and transferring to robotic hands without robot tactile data.
By Minseong Kweon, Qiao Feng, Yiming Huang, Lingjie Liu
Estimating physical pressure from vision is essential for understanding contact-rich hand-object interaction. However, prior vision-based pressure estimation methods are largely limited to planar surfaces and single image input, making them difficult to apply to dynamic hand-object interaction with diverse objects.