← Back to all news
arXiv Machine Learning

FlowTouch: View-Invariant Visuo-Tactile Prediction

FlowTouch is a model that predicts tactile sensor outputs from visual data in a view‑invariant manner by leveraging an object’s local 3D mesh to encode rich information while abstracting scene‑dependent details. It integrates scene reconstruction with Flow Matching‑based image generation, enabling it to bridge the sim‑to‑real gap and generalize to new sensor instances. The predicted tactile images can be used for downstream tasks such as grasp stability prediction.

Related stories

arXiv Machine Learning
Sep 10

ControlTac: Scaling Tactile Data with Physically Controlled Tactile Image Generation

ControlTac is a two‑stage framework that generates realistic tactile images conditioned on a single reference image, contact force, and contact pose. By incorporating these physical priors, it produces realistic samples across different sensors and captures task‑relevant variations. Experiments in object insertion, imitation learning, and object weighting show that datasets augmented with ControlTac consistently improve performance in dynamic real‑world settings.

By Dongyu Luo, Kelin Yu, Amir-Hossein Shahidzadeh, Cornelia Ferm\"uller, Yiannis Aloimonos, Ruohan Gao
diffusionrobotics
More like this →
arXiv Computer Vision
Oct 28

Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation

DreamTacVLA is a Vision‑Language‑Action framework that incorporates high‑resolution tactile images alongside wrist‑camera and third‑person vision to enable robots to anticipate and react to contact dynamics. It uses a Hierarchical Spatial Alignment loss to align tactile tokens with visual counterparts and a tactile world model to predict future tactile signals, trained on a hybrid dataset from digital twins and real experiments. The approach achieves up to 95% success on contact‑rich manipulation tasks, outperforming existing VLA baselines.

By Guo Ye, Zexi Zhang, Xu Zhao, Shang Wu, Haoran Lu, Shihan Lu, Han Liu
agentsroboticsmultimodalbenchmarkssafety
More like this →
arXiv Computer Vision
Oct 8

TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning

arXiv:2610.10288v1 Announce Type: new Abstract: Large-scale egocentric human interaction data is becoming an important source of physical supervision for embodied learning, yet video alone leaves the...

By Dayou Li, Hao Wang, Qianqian Yang, Zihao Zhu, Haoquan Fang, Ziyao Zeng, Yan Han, Zihan Wang, Yan Wang, Baoru Huang, Dilin Wang, Kenji Shimada, Yiyue Luo, Manling Li, Teresa Lv, Mustafa Mukadam, Rakesh Ranjan, Ruohan Zhang, Qi He, Changliu Liu, Xu Chen, Marco Pavone, Bangya Liu, Jiachen Li, Masayoshi Tomizuka, Zhiwen Fan
roboticsbenchmarks
More like this →