arXiv AI September 10, 2026 By Ziqin Huang, Yingyue Li, Chenyangguang Zhang, Ruida Zhang, Yuxin Chen, Gu Wang, Xingyu Liu, Masayoshi Tomizuka, Xiangyang Ji

3DWay: Generalizing Robot Manipulation via 3D Consistent Waypoints

Read the original on arXiv AI →

The paper introduces 3DWay, a method that predicts 3D-consistent waypoints for robot manipulation by first generating multi‑view consistent 2D waypoints and then triangulating them. This approach addresses the 3D ambiguity inherent in 2D trajectory predictions and leverages pretrained vision‑language models to provide explicit 3D motion specifications. Experiments demonstrate that 3DWay improves 3D spatial grounding and vision‑language reasoning, enhancing generalization for robot manipulation tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Sep 8

3DWay: Generalizing Robot Manipulation via 3D Consistent Waypoints

The paper introduces 3DWay, a method that predicts 3D consistent waypoints for robot manipulation using multi‑view images. By first generating 2D waypoints that are consistent across views and then triangulating them, the approach provides explicit 3D motion specifications while leveraging pretrained vision‑language models. Experiments demonstrate that 3DWay improves 3D spatial grounding and vision‑language reasoning, enhancing the generalization of robot manipulation policies.

llms robotics multimodal

arXiv AI

Jul 1

3D HAMSTER: Bridging Planning and Control in Hierarchical Vision Language Action Models through 3D Trajectory Guidance

arXiv:2606. 31329v1 Announce Type: cross Abstract: Hierarchical Vision-Language-Action (VLA) models decouple high-level planning from low-level control to improve generalization in robot manipulation.

By Dongyoon Hwang, Byungkun Lee, Dongjin Kim, Hyojin Jang, Hoiyeong Jin, Jueun Mun, Minho Park, Hojoon Lee, Hyunseung Kim, Jaegul Choo

llms robotics multimodal benchmarks

arXiv AI

Jul 7

Geometry-Aware Motion Latents for Learning Robust Manipulation Policies

arXiv:2607. 04714v1 Announce Type: cross Abstract: Learning motion latents for robotic manipulation heavily relies on extracting motion patterns from visual sequences, yet effective action abstractions require understanding three-dimensional geometric transformations.

By Yunchao Zhang, Yijia Weng, Ruizhe Liu, Ming Hu, Leonidas Guibas, Yanchao Yang

robotics benchmarks

arXiv Machine Learning

Jun 4

PointAction: 3D Points as Universal Action Representations for Robot Control

arXiv:2606. 03943v1 Announce Type: cross Abstract: Video-Action Models (VAMs) leverage the broad visual dynamics captured by pre-trained video diffusion models, offering a promising path toward generalizable robot manipulation.

By Mutian Tong, Han Jiang, Qiao Feng, Lingjie Liu, Jiatao Gu

diffusion robotics benchmarks

arXiv Computer Vision

Sep 2

Dyn-3D: Unveiling and Resolving Ego-Motion Ambiguity in Vision-Language Models

arXiv:2609.01059v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) tackle dynamic 3D spatial reasoning, ego-motion perception becomes essential to resolve monocular scale ambiguity. How...

By Jiayu Ding, Zhuodong Liu, Lei Zhang, Manyu Xiong, Hongbo Jin, Haoran Tang, Hongbo Zhang, Changen Zhu, Wenbo Xing

llms rag multimodal benchmarks

arXiv AI

Jul 14

Action Map Policy: Learning 3D Closed-loop Manipulation via Pixel Classification

arXiv:2607. 10706v1 Announce Type: cross Abstract: The action space poses a major challenge in robot learning, since it is often high-dimensional, can span long time horizons, and frequently admits multi-modal optimal solutions.

By Haojie Huang, Zhang Ye, Linfeng Zhao, Boce Hu, Mingxi Jia, Yu Qi, Ahmed Agha, Dian Wang, Robert Platt, Robin Walters

llms diffusion robotics multimodal

One email a morning, machine-written

Related stories

3DWay: Generalizing Robot Manipulation via 3D Consistent Waypoints

3D HAMSTER: Bridging Planning and Control in Hierarchical Vision Language Action Models through 3D Trajectory Guidance

Geometry-Aware Motion Latents for Learning Robust Manipulation Policies

PointAction: 3D Points as Universal Action Representations for Robot Control

Dyn-3D: Unveiling and Resolving Ego-Motion Ambiguity in Vision-Language Models

Action Map Policy: Learning 3D Closed-loop Manipulation via Pixel Classification