arXiv Computation and Language

Language-Guided Representation Learning for Robust Cross-Sensor Material Recognition

arXiv AI
Sep 24

BIDETA: Brain-Inspired Data-Efficient Tactile Adaptation for Unseen Sensors

BIDETA is a gradient‑free framework that adapts pretrained tactile models to new sensors using only a few labeled target contacts. It preserves pretrained representations while repairing sensor‑dependent feature neighborhoods through rapid support memory, support‑conditioned spectral graphs, and reliability‑gated recurrence. Experiments on multiple datasets show that BIDETA dramatically improves accuracy and speeds up adaptation compared to prior methods.

By Boheng Liu, Lan Wei, Ziyu Li, Chenghua Duan, Qing Li, Dandan Zhang, Xia Wu
arXiv AI
Sep 10

BIFTA: Brain-Inspired Few-Shot Tactile Adaptation for Unknown Sensors

The paper introduces BIFTA, a Brain‑Inspired Few‑Shot Tactile Adaptation framework that enables a frozen encoder to adapt quickly to an unknown tactile sensor using only a small labeled support set. It preserves pretrained representations via dual‑view statistical memory, builds support‑conditioned spectral graphs to correct sensor‑dependent feature neighborhoods, and employs uncertainty‑gated recurrent propagation to reinforce reliable cross‑query evidence. Benchmarks on three tactile datasets demonstrate that BIFTA dramatically improves adaptation performance, achieving an 87.09% mean Sparsh accuracy on SITR with just 10% labeled data—an increase of 47.22 percentage points over the best prior method.

By Boheng Liu, Ziyu Li, Xia Wu
arXiv Machine Learning
Sep 10

ControlTac: Scaling Tactile Data with Physically Controlled Tactile Image Generation

ControlTac is a two‑stage framework that generates realistic tactile images conditioned on a single reference image, contact force, and contact pose. By incorporating these physical priors, it produces realistic samples across different sensors and captures task‑relevant variations. Experiments in object insertion, imitation learning, and object weighting show that datasets augmented with ControlTac consistently improve performance in dynamic real‑world settings.

By Dongyu Luo, Kelin Yu, Amir-Hossein Shahidzadeh, Cornelia Ferm\"uller, Yiannis Aloimonos, Ruohan Gao
arXiv Machine Learning
4d ago

FlowTouch: View-Invariant Visuo-Tactile Prediction

FlowTouch is a model that predicts tactile sensor outputs from visual data in a view‑invariant manner by leveraging an object’s local 3D mesh to encode rich information while abstracting scene‑dependent details. It integrates scene reconstruction with Flow Matching‑based image generation, enabling it to bridge the sim‑to‑real gap and generalize to new sensor instances. The predicted tactile images can be used for downstream tasks such as grasp stability prediction.

By Seongjin Bien, Carlo Kneissl, Tobias J\"ulg, Frank Fundel, Thomas Ressler-Antal, Florian Walter, Bj\"orn Ommer, Gitta Kutyniok, Wolfram Burgard
arXiv Computer Vision
Oct 8

TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning

arXiv:2610.10288v1 Announce Type: new Abstract: Large-scale egocentric human interaction data is becoming an important source of physical supervision for embodied learning, yet video alone leaves the...

By Dayou Li, Hao Wang, Qianqian Yang, Zihao Zhu, Haoquan Fang, Ziyao Zeng, Yan Han, Zihan Wang, Yan Wang, Baoru Huang, Dilin Wang, Kenji Shimada, Yiyue Luo, Manling Li, Teresa Lv, Mustafa Mukadam, Rakesh Ranjan, Ruohan Zhang, Qi He, Changliu Liu, Xu Chen, Marco Pavone, Bangya Liu, Jiachen Li, Masayoshi Tomizuka, Zhiwen Fan
arXiv Computer Vision
Oct 21

Touch-JEPA: Tactile Representation Learning from Hand-Object Video with Limited Tactile Supervision

Touch-JEPA is a framework that learns tactile representations from hand‑object videos using limited tactile supervision. It first trains a self‑supervised model that predicts masked visual features at hand‑surface locations, capturing latent contact dynamics. The model is then jointly fine‑tuned on a small labeled subset, improving contact discrimination and tactile magnitude estimation on human‑hand benchmarks and transferring to robotic hands without robot tactile data.

By Minseong Kweon, Qiao Feng, Yiming Huang, Lingjie Liu
arXiv Machine Learning
Oct 9

TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learning

TACROSS is a low‑cost, scalable system that enables robots to learn from human touch by aligning tactile streams at the level of contact events rather than raw sensor values. It combines a $10.86 piezoresistive glove with 285 sensing points and a temporal Transformer that maps heterogeneous signals into a shared 256‑dimensional latent space. The approach uses robot demonstrations for action supervision while human demonstrations aid tactile representation learning, achieving a 3.5‑fold efficiency improvement and a 95.7% reduction in demonstration equipment cost across four manipulation tasks.

By Bo Chen, Huanzhang Hu, Junyang Ma, Bo Yue, Fangdi Yu, Haijier Chen, Xianxin Lai, Shuyu Pan, Zhen Yang, Xiaoquan Sun, Wenze Cui, Zhongliang Jiang, Shaopeng Liu, Jiayu Chen