arXiv Computer Vision
DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
5d ago
VTAM: Efficient Visuo-Tactile World-Action Models for Contact-Rich Manipulation without Tactile Pretraining
VTAM is a visuo‑tactile world‑action model that enables efficient contact‑rich manipulation without requiring tactile or action pretraining. It shares a frozen video VAE and pretrained transformer between RGB and tactile inputs, then fine‑tunes a randomly initialized action expert to predict actions, proprioception, and a flow‑force proxy derived from tactile optical flow. Trained solely on task demonstrations, VTAM outperforms baselines on real‑robot and simulation tasks, runs significantly faster than comparable models, and transfers zero‑shot across different tactile sensors.
arXiv Computer Vision
Oct 8
TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning
arXiv:2610.10288v1 Announce Type: new Abstract: Large-scale egocentric human interaction data is becoming an important source of physical supervision for embodied learning, yet video alone leaves the...
arXiv AI
Sep 30
Unified Visual-Tactile-Action Modeling from Human Demonstrations for Dexterous Manipulation
arXiv:2609.34182v2 Announce Type: replace-cross Abstract: Dexterous manipulation requires tactile feedback. However, robot tactile demonstrations are difficult to scale,because dexterous-hand teleope...
arXiv Computer Vision
Oct 28
Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation
DreamTacVLA is a Vision‑Language‑Action framework that incorporates high‑resolution tactile images alongside wrist‑camera and third‑person vision to enable robots to anticipate and react to contact dynamics. It uses a Hierarchical Spatial Alignment loss to align tactile tokens with visual counterparts and a tactile world model to predict future tactile signals, trained on a hybrid dataset from digital twins and real experiments. The approach achieves up to 95% success on contact‑rich manipulation tasks, outperforming existing VLA baselines.
arXiv Machine Learning
Sep 18
Agile-WAM: An Agile Tactile World Action Model for Contact-Rich Robot Control
Agile-WAM is a tactile World Action Model that jointly predicts future visual and tactile states and robot actions for contact‑rich manipulation. It encodes visual and tactile observations into a shared latent space and uses a vision‑tactile‑to‑action flow‑matching process to generate action chunks and future latents. The model introduces multi‑horizon multimodal prediction, leveraging the different timescales of vision and touch, and achieves a 29.4 % improvement in real‑world success rates with 11.9 ms inference latency across nine simulated and five real‑world tasks.
arXiv Computer Vision
Sep 18
DexTouch-WM: Learning Action-Conditioned Tactile World Models from Human Touch for Dexterous Robot Manipulation
DexTouch-WM is an action‑conditioned world model that learns from scalable human touch to predict future RGB observations and bilateral tactile dynamics for dexterous robot manipulation. By using compatible piezoresistive arrays on both human and robot hands and retargeting human motion into the robot action space, the model can be supervised with human interaction data while keeping a fixed amount of real‑robot supervision. Experiments show that adding up to 100 hours of human interaction improves robot‑domain visual, geometric, and contact prediction, and the model can serve as a surrogate environment for policy evaluation and synthetic trajectory generation.