arXiv AI

UniTac: A Unified Multimodal Model for Cross-Sensor Tactile Understanding and Generation

arXiv:2606. 31451v1 Announce Type: cross Abstract: Unified multimodal models (UMMs) have shown great promise in integrating understanding and generation across diverse modalities.

arXiv Machine Learning
Sep 10

ControlTac: Scaling Tactile Data with Physically Controlled Tactile Image Generation

ControlTac is a two‑stage framework that generates realistic tactile images conditioned on a single reference image, contact force, and contact pose. By incorporating these physical priors, it produces realistic samples across different sensors and captures task‑relevant variations. Experiments in object insertion, imitation learning, and object weighting show that datasets augmented with ControlTac consistently improve performance in dynamic real‑world settings.

By Dongyu Luo, Kelin Yu, Amir-Hossein Shahidzadeh, Cornelia Ferm\"uller, Yiannis Aloimonos, Ruohan Gao
arXiv AI
Jul 7

TacReasoner: A Dynamic Tactile-Language Framework for Interactive Reasoning in Real-World Scenarios

arXiv:2607. 05131v1 Announce Type: new Abstract: Among the five primary human senses, tactile is arguably the most fundamental to survival, as it enables the perception of physical contact and interaction in real-world environments.

By Kailin Lyu, Di Wu, Long Xiao, Jianning Zeng, Jianwei He, Chang Lin, Lianyu Hu, Lin Shu, Jie Hao, Ce Hao
Hugging Face Trending Papers
Jul 6

TacReasoner: A Dynamic Tactile-Language Framework for Interactive Reasoning in Real-World Scenarios

Among the five primary human senses, tactile is arguably the most fundamental to survival, as it enables the perception of physical contact and interaction in real-world environments. In this paper, we explore two key challenges of integrating tactile sensing into intelligent systems for multimodal reasoning: (i) insufficient modeling of dynamic tactile signals, which restricts reasoning over temporally evolving properties, and (ii) hallucination in tactile foundation models caused by the absence of explicit reasoning mechanisms, leading to unstable real-world inference.

arXiv AI
Sep 24

BIDETA: Brain-Inspired Data-Efficient Tactile Adaptation for Unseen Sensors

BIDETA is a gradient‑free framework that adapts pretrained tactile models to new sensors using only a few labeled target contacts. It preserves pretrained representations while repairing sensor‑dependent feature neighborhoods through rapid support memory, support‑conditioned spectral graphs, and reliability‑gated recurrence. Experiments on multiple datasets show that BIDETA dramatically improves accuracy and speeds up adaptation compared to prior methods.

By Boheng Liu, Lan Wei, Ziyu Li, Chenghua Duan, Qing Li, Dandan Zhang, Xia Wu
arXiv Computer Vision
Aug 24

VT-MUSE: Multimodal Unified Sequential Visuotactile Representation Learning for Manipulation

VT-MUSE is a multimodal unified sequential representation learning framework for visuotactile manipulation. It uses a two‑stage approach: first, modality‑specific encoders are jointly adapted with cross‑modal temporal alignment and masked‑view consistency; second, a conditional variational latent model processes masked visual sequences and full tactile histories, with auxiliary decoders reconstructing recent visual observations and predicting tactile depth changes. The resulting representation is fed into a lightweight Transformer policy via gated cross‑attention, achieving an 11‑percentage‑point improvement over the strongest baseline in simulation and significant gains in real‑world experiments.

By Congsheng Xu, Qiaochu Yang, Fangyuan Shi, Yifan Han, Baijun Chen, Yiming Wang, Haonan Zhao, Daolin Ma, Xiaokang Yang, Hesheng Wang