arXiv Machine Learning

Learning tactile perception from high-bandwidth single-point sensing

arXiv AI
4d ago

Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors

Tactile-JEPA is a self‑supervised pre‑training method for distributed tactile sensors that leverages the sensors’ spatial topology to learn topology‑aware representations. It predicts embeddings of masked sensing elements using a sensor connectivity graph and dual‑scale masking to capture both local contact details and the global tactile surface state. Evaluated on three diverse datasets, it improves force estimation by 6.3 % and in‑hand orientation error by 20.8 % over previous state‑of‑the‑art methods, and yields consistent gains in downstream tasks such as policy learning.

By Elizaveta Kovtun, Matvey Konovalov, Andrey Sakhovskiy, Semen Budennyy
arXiv AI
Sep 10

BIFTA: Brain-Inspired Few-Shot Tactile Adaptation for Unknown Sensors

The paper introduces BIFTA, a Brain‑Inspired Few‑Shot Tactile Adaptation framework that enables a frozen encoder to adapt quickly to an unknown tactile sensor using only a small labeled support set. It preserves pretrained representations via dual‑view statistical memory, builds support‑conditioned spectral graphs to correct sensor‑dependent feature neighborhoods, and employs uncertainty‑gated recurrent propagation to reinforce reliable cross‑query evidence. Benchmarks on three tactile datasets demonstrate that BIFTA dramatically improves adaptation performance, achieving an 87.09% mean Sparsh accuracy on SITR with just 10% labeled data—an increase of 47.22 percentage points over the best prior method.

By Boheng Liu, Ziyu Li, Xia Wu
arXiv AI
Jul 7

OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

arXiv:2607. 03723v1 Announce Type: cross Abstract: Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact-rich manipulation, where success significantly depends on local force and contact geometry.

By Kelin Yu, Haode Zhang, Harish Ravichandar, Yunhai Han, Ruohan Gao
arXiv Computer Vision
Sep 18

DexTouch-WM: Learning Action-Conditioned Tactile World Models from Human Touch for Dexterous Robot Manipulation

DexTouch-WM is an action‑conditioned world model that learns from scalable human touch to predict future RGB observations and bilateral tactile dynamics for dexterous robot manipulation. By using compatible piezoresistive arrays on both human and robot hands and retargeting human motion into the robot action space, the model can be supervised with human interaction data while keeping a fixed amount of real‑robot supervision. Experiments show that adding up to 100 hours of human interaction improves robot‑domain visual, geometric, and contact prediction, and the model can serve as a surrogate environment for policy evaluation and synthetic trajectory generation.

By Yan Qin, Yue Chen, Wenwei Lin, Shujia Liu, Chuqiao Lyu, Kailun Su, Chenze Yu, Ping Luo, Wenbo Ding, Tianxing Chen, Renjing Xu
arXiv AI
3d ago

BIDETA: Brain-Inspired Data-Efficient Tactile Adaptation for Unseen Sensors

BIDETA is a gradient‑free framework that adapts pretrained tactile models to new sensors using only a few labeled target contacts. It preserves pretrained representations while repairing sensor‑dependent feature neighborhoods through rapid support memory, support‑conditioned spectral graphs, and reliability‑gated recurrence. Experiments on multiple datasets show that BIDETA dramatically improves accuracy and speeds up adaptation compared to prior methods.

By Boheng Liu, Lan Wei, Ziyu Li, Chenghua Duan, Qing Li, Dandan Zhang, Xia Wu
arXiv Machine Learning
Sep 10

ControlTac: Scaling Tactile Data with Physically Controlled Tactile Image Generation

ControlTac is a two‑stage framework that generates realistic tactile images conditioned on a single reference image, contact force, and contact pose. By incorporating these physical priors, it produces realistic samples across different sensors and captures task‑relevant variations. Experiments in object insertion, imitation learning, and object weighting show that datasets augmented with ControlTac consistently improve performance in dynamic real‑world settings.

By Dongyu Luo, Kelin Yu, Amir-Hossein Shahidzadeh, Cornelia Ferm\"uller, Yiannis Aloimonos, Ruohan Gao
arXiv Machine Learning
Sep 18

Agile-WAM: An Agile Tactile World Action Model for Contact-Rich Robot Control

Agile-WAM is a tactile World Action Model that jointly predicts future visual and tactile states and robot actions for contact‑rich manipulation. It encodes visual and tactile observations into a shared latent space and uses a vision‑tactile‑to‑action flow‑matching process to generate action chunks and future latents. The model introduces multi‑horizon multimodal prediction, leveraging the different timescales of vision and touch, and achieves a 29.4 % improvement in real‑world success rates with 11.9 ms inference latency across nine simulated and five real‑world tasks.

By Hanchu Zhou, Brendan Lynch, Raman Goyal, Dechen Gao, Begum Kasap, Boqi Zhao, Junshan Zhang