← Back to all news
arXiv Computer Vision By Minseong Kweon, Qiao Feng, Yiming Huang, Lingjie Liu

Touch-JEPA: Tactile Representation Learning from Hand-Object Video with Limited Tactile Supervision

Read the original on arXiv Computer Vision →

Touch-JEPA is a framework that learns tactile representations from hand‑object videos using limited tactile supervision. It first trains a self‑supervised model that predicts masked visual features at hand‑surface locations, capturing latent contact dynamics. The model is then jointly fine‑tuned on a small labeled subset, improving contact discrimination and tactile magnitude estimation on human‑hand benchmarks and transferring to robotic hands without robot tactile data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computer Vision
Oct 8

TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning

arXiv:2610.10288v1 Announce Type: new Abstract: Large-scale egocentric human interaction data is becoming an important source of physical supervision for embodied learning, yet video alone leaves the...

By Dayou Li, Hao Wang, Qianqian Yang, Zihao Zhu, Haoquan Fang, Ziyao Zeng, Yan Han, Zihan Wang, Yan Wang, Baoru Huang, Dilin Wang, Kenji Shimada, Yiyue Luo, Manling Li, Teresa Lv, Mustafa Mukadam, Rakesh Ranjan, Ruohan Zhang, Qi He, Changliu Liu, Xu Chen, Marco Pavone, Bangya Liu, Jiachen Li, Masayoshi Tomizuka, Zhiwen Fan
roboticsbenchmarks
More like this →
Hugging Face Trending Papers
Sep 17

TouchSight: Bare-Handed Tactile Prediction from Egocentric Video via Generative Visual Augmentation

TouchSight is a monocular egocentric vision framework that predicts dense full-hand contact forces without tactile sensors. It uses 500 hours of pressure-glove data and a 20-hour TwinTouch-20H dataset where generative models render gloved recordings as bare-hand videos, bridging the appearance gap. The system outperforms previous methods on OakInk2, generalizes to unseen natural bare-hand egocentric videos, and improves as glove supervision increases.

robotics
More like this →
arXiv AI
Sep 18

TouchSight: Bare-Handed Tactile Prediction from Egocentric Video via Generative Visual Augmentation

TouchSight is a monocular egocentric vision framework that predicts dense full-hand contact forces from video. It uses 500 hours of pressure‑glove recordings and hand‑object interaction data, and introduces TwinTouch‑20H, a dataset of 20 hours of paired visual data where generative models render gloved recordings as bare‑hand observations while preserving tactile labels. The system outperforms prior methods on OakInk2, generalizes qualitatively to natural bare‑hand egocentric videos from unseen datasets, and improves consistently as glove supervision scales.

By Danyan Zhou, Jinxuan Lu, Jiawei Lin, Tianxing Chen, Chuqiao Lyu, Wenbo Ding
robotics
More like this →