arXiv Computer Vision

Proximity3D: Shape from Capacitive Proximity on Sensing Manifold

arXiv Machine Learning
6d ago

SoTa: Soft Tactile Skins for Dexterous Manipulation

SoTa is a low‑cost, capacitive tactile skin that offers full‑hand coverage for both humans and robots, featuring a shared layout of 202 taxels across finger and palm regions. The multilayer design with fabric electrodes allows in‑house fabrication for under $10 in materials, and the sensor maintains over 97% of its response after 10,000 loading‑unloading cycles and 1,280 tight‑fist folds. Using SoTa, human‑robot co‑training improves in‑distribution success on three contact‑rich manipulation tasks, and adding human demonstrations more than doubles mean success across eight evaluation conditions, from 22.8% to 45.9%, including all out‑of‑distribution scenarios.

By Jingyun Yang, Baiyu Shi, Timothy Yu, Haitian Liu, Alberta Longhini, Weichen Wang, Rika Antonova, Zhenan Bao, Jeannette Bohg
arXiv Computer Vision
Aug 31

Object Reconstruction under Occlusion with Generative Priors and Contact-induced Constraints

The paper presents a method for reconstructing object geometry under occlusion by combining generative shape priors with contact-based constraints. Generative models provide plausible guesses for unseen parts, while contact information from videos or physical interactions supplies sparse boundary constraints. The authors integrate these cues through a contact-guided 3D generation framework, demonstrating improved reconstruction on synthetic and real-world datasets compared to baseline approaches.

By Minghan Zhu, Zhiyi Wang, Qihang Sun, Maani Ghaffari, Michael Posa
arXiv Machine Learning
2d ago

TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learning

TACROSS is a low‑cost, scalable system that enables robots to learn from human touch by aligning tactile streams at the level of contact events rather than raw sensor values. It combines a $10.86 piezoresistive glove with 285 sensing points and a temporal Transformer that maps heterogeneous signals into a shared 256‑dimensional latent space. The approach uses robot demonstrations for action supervision while human demonstrations aid tactile representation learning, achieving a 3.5‑fold efficiency improvement and a 95.7% reduction in demonstration equipment cost across four manipulation tasks.

By Bo Chen, Huanzhang Hu, Junyang Ma, Bo Yue, Fangdi Yu, Haijier Chen, Xianxin Lai, Shuyu Pan, Zhen Yang, Xiaoquan Sun, Wenze Cui, Zhongliang Jiang, Shaopeng Liu, Jiayu Chen
arXiv AI
Sep 23

Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors

Tactile-JEPA is a self‑supervised pre‑training method for distributed tactile sensors that leverages the sensors’ spatial topology to learn topology‑aware representations. It predicts embeddings of masked sensing elements using a sensor connectivity graph and dual‑scale masking to capture both local contact details and the global tactile surface state. Evaluated on three diverse datasets, it improves force estimation by 6.3 % and in‑hand orientation error by 20.8 % over previous state‑of‑the‑art methods, and yields consistent gains in downstream tasks such as policy learning.

By Elizaveta Kovtun, Matvey Konovalov, Andrey Sakhovskiy, Semen Budennyy
arXiv Computer Vision
Aug 28

Reconstructing Humans and Objects in Interaction using Large Reconstruction Models

The paper introduces MILO, a framework that uses Large Reconstruction Models (LRMs) to reconstruct detailed 3D human‑object interactions from a single image. By treating the LRM mesh as a geometric scaffold, MILO segments it into human and object parts, fits a parametric body model to the human component, and optionally aligns an object template to the object component. The approach achieves higher reconstruction accuracy than existing baselines across multiple benchmarks and interaction scenarios.

By Agniv Chatterjee, Georgios Pavlakos