arXiv AI
DepthWorld: 3D World Model for Robot Manipulation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Oct 6
DepthWorld: 3D World Model for Robot Manipulation
World models offer a data-driven alternative to traditional simulators for robotics, with applications spanning policy evaluation, improvement, and planning. All of these uses depend on faithful 3D ge...
arXiv Machine Learning
Oct 5
XGenAct: Geometry-Enhanced World Action Models through Cross-Task Generation
XGenAct is a world action model that encodes RGB observations, robot actions, metric depth, surface normals, and functional role segmentation as RGB videos using deterministic codecs. It trains a single video diffusion transformer with a unified objective, sampling different perception and action tasks to learn temporal prediction across multiple spatial modalities without separate heads. Experiments on RLBench show that structured perception training boosts closed‑loop success, with XGenAct achieving 52% success on five external tasks compared to 26% for the best baselines, and it outperforms pipelines that first generate RGB and then apply a frozen perception expert for depth and segmentation prediction.
arXiv Computer Vision
Sep 17
PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics
arXiv:2609.19142v1 Announce Type: new Abstract: World models endow perceptual systems with the ability to predict how scenes evolve under interaction. They are most beneficial when trained on diverse...
arXiv Computer Vision
Sep 3
Spatially Aware World Action Model via Geometric Latent Diffusion
The paper introduces Spatially Aware World Action Model (SA‑WAM), a diffusion‑based framework that extends existing World Action Models by incorporating depth information alongside RGB to enable 3‑D‑aware action and future‑state prediction. SA‑WAM repurposes a pretrained video diffusion model, using a nonlinear encoding to map unbounded depth into the tokenizer’s bounded domain, thus preserving pretrained visual priors without 3‑D‑specific fine‑tuning. The model achieves state‑of‑the‑art performance on RoboCasa and LIBERO‑Plus benchmarks and demonstrates superior real‑world performance on a UR5 robotic arm in randomized environments, while also providing analysis linking world‑model prediction quality to rollout success.
arXiv Machine Learning
1d ago
StereoEngine: Scaling Stereo Bimanual Robot Data without Stereo Cameras
StereoEngine introduces two methods for generating realistic synthetic stereo data for robotic manipulation. The first method warps a monocular video into a right view using depth estimation and a video diffusion model to inpaint disoccluded regions, while the second method renders stereo pairs directly in simulation. These approaches outperform existing baselines in stereo quality and enable improved bimanual manipulation policies when used for mid‑training or task‑specific fine‑tuning.
Hugging Face Trending Papers
Aug 6
LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models
World models enable agents to perform forward rollout and planning without real-world interaction. However, their application in open-world embodied intelligence remains limited by the high cost of action annotations and the heterogeneity of action spaces across platforms.