arXiv AI By Ruixing Jin, Zicheng Zhu, Ruixiang Ouyang, Sheng Xu, Bo Yue, Zhizheng Wu, Guiliang Liu

Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action Models

Read the original on arXiv AI →

arXiv:2603. 22876v2 Announce Type: replace-cross Abstract: Learning a generalist control policy for robotic manipulation typically relies on large-scale datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 25

KeyGen: Unsupervised Keypoint based Object-Centric Representations for Category-Level Policy Generalization

KeyGen is a framework that learns canonical 3D keypoints from point clouds to create structured, object‑centric representations for policy learning in robotic manipulation. By conditioning a visuomotor diffusion policy on these keypoints and object geometry, it predicts full manipulation trajectories that maintain geometric correspondence across different object instances. Experiments on a photorealistic simulation benchmark with three tasks show that KeyGen outperforms prior methods on both seen and unseen objects, scales with more demonstrations, remains robust to rescaling, and performs well in real‑world manipulation.

By Shuxin Cao, Liquan Wang, Masoud Moghani, Benjamin Joffe, Animesh Garg
arXiv Computer Vision
2d ago

ForeAct3D: Policy-Grounded Future World Modeling for VLA Policies

ForeAct3D is a framework that grounds future world modeling in Vision‑Language‑Action policies by decoding depth, semantic segmentation, and camera pose from policy representations into current and future 3D scene states. The future queries are conditioned on the policy‑generated action chunk, and a physical‑consistency closure enforces background staticity, instance‑level rigidity, and wrist‑camera pose alignment with end‑effector kinematics. This approach eliminates the need for future prediction at inference and achieves high success rates on LIBERO and CALVIN, outperforming the base policy across all suites.

By Zhe Tao, Feiran Wang, Gaowen Liu, Ramana Rao Kompella, Yan Yan
arXiv AI
Jun 2

From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data

arXiv:2606. 00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models.

By Zhiyuan Feng, Qixiu Li, Huizhi Liang, Rushuai Yang, Yichao Shen, Zhiying Du, Zhaowei Zhang, Yu Deng, Li Zhao, Hao Zhao, Zongqing Lu, Oier Mees, Marc Pollefeys, Jiaolong Yang, Baining Guo