arXiv Machine Learning

ControlTac: Scaling Tactile Data with Physically Controlled Tactile Image Generation

ControlTac is a two‑stage framework that generates realistic tactile images conditioned on a single reference image, contact force, and contact pose. By incorporating these physical priors, it produces realistic samples across different sensors and captures task‑relevant variations. Experiments in object insertion, imitation learning, and object weighting show that datasets augmented with ControlTac consistently improve performance in dynamic real‑world settings.

arXiv Machine Learning

4d ago

FlowTouch: View-Invariant Visuo-Tactile Prediction

FlowTouch is a model that predicts tactile sensor outputs from visual data in a view‑invariant manner by leveraging an object’s local 3D mesh to encode rich information while abstracting scene‑dependent details. It integrates scene reconstruction with Flow Matching‑based image generation, enabling it to bridge the sim‑to‑real gap and generalize to new sensor instances. The predicted tactile images can be used for downstream tasks such as grasp stability prediction.

By Seongjin Bien, Carlo Kneissl, Tobias J\"ulg, Frank Fundel, Thomas Ressler-Antal, Florian Walter, Bj\"orn Ommer, Gitta Kutyniok, Wolfram Burgard

diffusion robotics

arXiv AI

Sep 30

Unified Visual-Tactile-Action Modeling from Human Demonstrations for Dexterous Manipulation

arXiv:2609.34182v2 Announce Type: replace-cross Abstract: Dexterous manipulation requires tactile feedback. However, robot tactile demonstrations are difficult to scale,because dexterous-hand teleope...

By Wenqiao Li, Qianyou Zhao, Jiawen Hao, Xuezhou Zhu, Tengyu Liu, Kaifeng Zhang, Chuan Wen, Siyuan Huang

robotics benchmarks

arXiv AI

Jul 1

UniTac: A Unified Multimodal Model for Cross-Sensor Tactile Understanding and Generation

arXiv:2606. 31451v1 Announce Type: cross Abstract: Unified multimodal models (UMMs) have shown great promise in integrating understanding and generation across diverse modalities.

By Jiahang Tu, Fengyu Yang, Chenyang Ma, Xihang Yu, Ziyao Zeng, Shaokai Wu, Hanbin Zhao, Zhi Tao, Chao Zhang, Hui Qian, Alex Wong

multimodal benchmarks safety

arXiv Computer Vision

Oct 19

EgoTac: In-the-wild Tactile Prediction from Egocentric Vision

arXiv:2608.15060v2 Announce Type: replace Abstract: Touch is fundamental to dexterous manipulation, yet most egocentric human data increasingly used for robot learning lacks tactile information. Dire...

By Wenkang Zhang, Chengbo Yuan, Zicheng Zhang, Zhengxue Cheng, Yang Gao

robotics benchmarks

arXiv Computer Vision

Oct 8

TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning

arXiv:2610.10288v1 Announce Type: new Abstract: Large-scale egocentric human interaction data is becoming an important source of physical supervision for embodied learning, yet video alone leaves the...

By Dayou Li, Hao Wang, Qianqian Yang, Zihao Zhu, Haoquan Fang, Ziyao Zeng, Yan Han, Zihan Wang, Yan Wang, Baoru Huang, Dilin Wang, Kenji Shimada, Yiyue Luo, Manling Li, Teresa Lv, Mustafa Mukadam, Rakesh Ranjan, Ruohan Zhang, Qi He, Changliu Liu, Xu Chen, Marco Pavone, Bangya Liu, Jiachen Li, Masayoshi Tomizuka, Zhiwen Fan

robotics benchmarks

arXiv Machine Learning

Sep 22

$N_0$-Foundation: Towards the Age of Tactile Intelligence

arXiv:2608.29601v2 Announce Type: replace-cross Abstract: We present $N_0$-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale mul...

By NeoteAI Team, Fudan TEAI Team

robotics multimodal benchmarks

arXiv AI

Oct 21

Learning Tactile-Aware Hand Representations from Human Hand-Object Interaction Data

arXiv:2610.27995v1 Announce Type: cross Abstract: Tactile representation learning for dexterous manipulation often requires substantial robot-side data collection, limiting its scalability across dif...

By Yeonguk Yu, Jaehong Kim, Byungok Han

robotics

arXiv Machine Learning

Sep 1

$\mathcal{N}_0$-Foundation: Towards the Age of Tactile Intelligence

arXiv:2608.29601v1 Announce Type: cross Abstract: We present $\mathcal{N}_0$-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale m...

By NeoteAI Team, Fudan TEAI Team

robotics multimodal benchmarks

arXiv Computer Vision

Oct 28

Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation

DreamTacVLA is a Vision‑Language‑Action framework that incorporates high‑resolution tactile images alongside wrist‑camera and third‑person vision to enable robots to anticipate and react to contact dynamics. It uses a Hierarchical Spatial Alignment loss to align tactile tokens with visual counterparts and a tactile world model to predict future tactile signals, trained on a hybrid dataset from digital twins and real experiments. The approach achieves up to 95% success on contact‑rich manipulation tasks, outperforming existing VLA baselines.

By Guo Ye, Zexi Zhang, Xu Zhao, Shang Wu, Haoran Lu, Shihan Lu, Han Liu

agents robotics multimodal benchmarks safety

arXiv AI

Jul 7

OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

arXiv:2607. 03723v1 Announce Type: cross Abstract: Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact-rich manipulation, where success significantly depends on local force and contact geometry.

By Kelin Yu, Haode Zhang, Harish Ravichandar, Yunhai Han, Ruohan Gao

agents reinforcement-learning robotics

arXiv Computer Vision

Oct 7

PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation

arXiv:2610.08068v1 Announce Type: new Abstract: Realistic physical interaction is a cornerstone of embodied intelligence, yet collecting paired visual--tactile data remains costly. Visual-to-tactile...

By Guo Tang, Yongtao Wang

llms diffusion computer-vision reinforcement-learning robotics

arXiv Computation and Language

Sep 15

Language-Guided Representation Learning for Robust Cross-Sensor Material Recognition

arXiv:2609.14783v1 Announce Type: cross Abstract: Robots need touch to manipulate objects safely and reliably, as many properties, such as softness, texture, and contact stability, are hard to infer...

By Mashood M. Mohsan, Muhayy Ud Din, Binzhao Xu, Ahmad Abubakar, Irfan Hussain

rag robotics efficiency benchmarks

Related stories