ActiveAlign is a framework that uses active perception to close the Real2Sim2Real gap in robot manipulation. It introduces two modules: ActiveAlign‑Waving, which performs hand‑eye calibration via active hand motions and differentiable rendering, and ActiveAlign‑Touching, which refines object alignment with tactile feedback for millimeter‑level precision. The approach enables strong real‑to‑simulation alignment, allowing simulation environments to be curated from uncalibrated datasets and enabling a real Unitree G1 robot to manipulate millimeter‑scale objects such as buttons and sliders.
arXiv:2609.37089v1 Announce Type: new
Abstract: Real-world videos provide rich demonstrations of manipulation, but turning them into reusable robot skills requires visually aligned environments, exec...
By Kerui Ren, Yingxiang Xu, Kaiwen Song, Linning Xu, Bo Dai, Mulin Yu, Tao Lu
arXiv:2609.24124v1 Announce Type: cross
Abstract: Active perception and manipulation are crucial for robots to interact with complex scenes. Existing benchmarks struggle to evaluate how robots effect...
By Yibo Li, Enshen Zhou, Rui Chen, Yanjun Ding, Mengzhen Liu, Yi Han, Jiabo Zhan, Lipeng Wang, Shanghang Zhang, Lu Sheng
AffordSim is a scalable data generator and benchmark that integrates open‑vocabulary 3D affordance prediction into simulation‑based trajectory generation for robotic manipulation. Given a natural‑language task description, it synthesizes task‑relevant scenes, queries affordances, grounds them on object surfaces, samples region‑conditioned grasps, and selects executable candidates with motion planning, while randomizing pose, texture, lighting, noise, and backgrounds for sim‑to‑real transfer. The benchmark includes 50 tasks, five robot embodiments, and over 500 objects, achieving 93% of the success rate of manual contact annotations on affordance‑critical tasks and 89% on hard composite tasks, with policies trained on AffordSim transferring zero‑shot to a real Franka FR3 with 24% average success.
By Mingyang Li, Haofan Xu, Haowen Sun, Xinzhe Chen, Sihua Ren, Liqi Huang, Xinyang Sui, Chenyang Miao, Jiawei Ye, Qiongjie Cui, Xingyu Chen, Zeyang Liu, Xuguang Lan
arXiv:2604. 01001v2 Announce Type: replace-cross Abstract: We introduce EgoSim, a closed-loop egocentric world simulator that generates spatially consistent interaction videos and persistently updates the underlying 3D scene state for continuous simulation.
By Jinkun Hao, Mingda Jia, Ruiyan Wang, Hongrui Zhu, Jiafei Cao, Xihui Liu, Ran Yi, Lizhuang Ma, Jiangmiao Pang, Xudong Xu
arXiv:2606. 08057v1 Announce Type: cross Abstract: Egocentric RGB-D videos offer a natural source of human dexterous manipulation demonstrations, but existing data is difficult to use for robot learning because object pose, geometry, and contact information are often missing or require pre-scanned object assets.
By Yichen Niu, Haoran Lv, Xinrui Zhang, Xueyao Wan, Shiyu Gao, Ying Ai, Hui Xu, Yongqi Hu, Hengyi Zhang, Yang Xie, Zhaxizhuoma, Yue Zhao, Zhenshan Bing, Yan Ding, Jianxing Liu
ActiveScale is a framework that enhances active perception for robots by integrating model, data, and hardware innovations. It augments vision‑language‑action models with historical video observations and explicit camera‑pose supervision, and introduces a scalable human‑robot mid‑training recipe using 1000 hours of egocentric and robotic data. The Active‑perception Mobile‑manipulation Platform (AMP) enables single‑operator teleoperation for scalable demonstration collection, leading to improved success rates on active‑perception tasks.
By Shuai Zhou, Kaisheng Pang, Wenxuan Song, Wenjie Zhang, Xinhu Zheng, Haoang Li
arXiv:2611.08372v1 Announce Type: cross
Abstract: Visuomotor policies trained through imitation learning are highly sensitive to changes in camera viewpoint. This is particularly problematic for mobi...
By Sateesh Kumar, Arnav Balaji, Arpit Bahety, Skand Peri, Georgios Pavlakos, Roberto Mart\'in-Mart\'in
arXiv:2609.24660v2 Announce Type: cross
Abstract: Human demonstrations offer a scalable way to collect manipulation data, but their contacts may be unstable or infeasible when transferred to a robot...
By Shengcheng Luo, Xiaoyang Cheng, Hong Ying, Xiaoying Zhou, Jiaming Jiang, Haoran Guo, Wanlin Li, Ziyuan Jiao, Chenxi Xiao
arXiv:2606. 11767v1 Announce Type: cross Abstract: Blind grasping with a dexterous hand is a crucial manipulation capability.
By Shengcheng Luo, Xiyan Huang, Zhe Xu, Wanlin Li, Ziyuan Jiao, Chenxi Xiao
arXiv:2604. 17787v2 Announce Type: replace-cross Abstract: Precision-critical manipulation requires both global trajectory organization and local execution correction, yet most vision-language-action (VLA) policies generate actions within a single unified space.
By Tingzheng Jia, Kan Guo, Lanping Qian, Yongli Hu, Daxin Tian, Guixian Qu, Chunmian Lin, Baocai Yin, Jiapu Wang
arXiv:2607. 25912v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have shown strong potential for general robot manipulation, but most existing models rely on 2D visual-language backbones and lack fine-grained 3D understanding of target objects, especially under occlusion, pose variation, scale changes, and precise spatial interaction.
By Zonghe Liu (University of Hong Kong), Shanyuan Jie (Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences), Xiaoquan Sun (Huazhong University of Science and Technology), Chen Cao (University of Hong Kong), Zetian Xu (University of Hong Kong), Zongsheng Liu (Beijing University of Aeronautics and Astronautics), Jiayu Chen (University of Hong Kong, Infiforce)