VTAM: Efficient Visuo-Tactile World-Action Models for Contact-Rich Manipulation without Tactile Pretraining

VTAM is a visuo‑tactile world‑action model that enables efficient contact‑rich manipulation without requiring tactile or action pretraining. It shares a frozen video VAE and pretrained transformer between RGB and tactile inputs, then fine‑tunes a randomly initialized action expert to predict actions, proprioception, and a flow‑force proxy derived from tactile optical flow. Trained solely on task demonstrations, VTAM outperforms baselines on real‑robot and simulation tasks, runs significantly faster than comparable models, and transfers zero‑shot across different tactile sensors.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• diffusion
• computer-vision
• robotics
• fine-tuning
• efficiency
• multimodal