arXiv AI
DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Sep 8
DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination
DeCAL is a vision‑language‑action model designed for dexterous manipulation that incorporates tactile sensing through adaptive visuo‑tactile fusion and latent co‑imagination. It uses a Mixture‑of‑Transformers architecture with specialized experts for understanding, imagination, and action, enabling efficient information flow and dynamic regulation of tactile inputs. Experiments show DeCAL achieves state‑of‑the‑art performance, with a 71% average success rate and 83.4% progress success rate, and generalizes well to unseen scenarios.
arXiv Machine Learning
Jul 16
Tactile Modality Fusion for Vision-Language-Action Models
arXiv:2603. 14604v2 Announce Type: replace-cross Abstract: We propose TacFiLM, a lightweight modality-fusion approach that integrates visual-tactile signals into vision-language-action (VLA) models.
arXiv Computer Vision
Oct 28
Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation
DreamTacVLA is a Vision‑Language‑Action framework that incorporates high‑resolution tactile images alongside wrist‑camera and third‑person vision to enable robots to anticipate and react to contact dynamics. It uses a Hierarchical Spatial Alignment loss to align tactile tokens with visual counterparts and a tactile world model to predict future tactile signals, trained on a hybrid dataset from digital twins and real experiments. The approach achieves up to 95% success on contact‑rich manipulation tasks, outperforming existing VLA baselines.
arXiv AI
Sep 30
Unified Visual-Tactile-Action Modeling from Human Demonstrations for Dexterous Manipulation
arXiv:2609.34182v2 Announce Type: replace-cross Abstract: Dexterous manipulation requires tactile feedback. However, robot tactile demonstrations are difficult to scale,because dexterous-hand teleope...
arXiv Computer Vision
Sep 22
DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation
arXiv:2609.24976v1 Announce Type: cross Abstract: Dexterous manipulation depends on contact dynamics that are often only partially observable from vision. Recent World-Action Models (WAMs) couple pre...
arXiv Machine Learning
Sep 18
Agile-WAM: An Agile Tactile World Action Model for Contact-Rich Robot Control
Agile-WAM is a tactile World Action Model that jointly predicts future visual and tactile states and robot actions for contact‑rich manipulation. It encodes visual and tactile observations into a shared latent space and uses a vision‑tactile‑to‑action flow‑matching process to generate action chunks and future latents. The model introduces multi‑horizon multimodal prediction, leveraging the different timescales of vision and touch, and achieves a 29.4 % improvement in real‑world success rates with 11.9 ms inference latency across nine simulated and five real‑world tasks.