Perceive, Predict, and Plan: a Unified World Action Model for Egocentric Dexterous Manipulation

The paper introduces P³-WAM, a unified world action model for egocentric dexterous manipulation that integrates 3D perception, action-conditioned prediction, and goal-directed planning. It operates in three modes—Perceive, Predict, and Plan—using a mixture-of-transformers architecture that shares base weights and adapts to modality-specific inputs. Experiments demonstrate that unified training improves performance across all three capabilities, outperforming specialized baselines on 3D hand reconstruction and action-conditioned video prediction while producing more instruction‑adherent manipulation videos.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

• llms
• diffusion
• robotics
• multimodal