arXiv AI
Embodied Zero-Shot Task Planning via Hardware-Aware LLM-VLM Integration for Grounded Robotic Manipulation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 2
Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
arXiv:2605. 30280v2 Announce Type: replace-cross Abstract: Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments.
arXiv AI
Sep 30
Gondola: Grounded Vision Language Planning for Robotic Manipulation
arXiv:2506.11261v2 Announce Type: replace-cross Abstract: Vision-language-action (VLA) models have shown promising progress in robotic manipulation. However, directly mapping visual observations and...
arXiv AI
Oct 1
HiWE: Hierarchical World Knowledge Model with Visual Keypoint Enhancement for Zero-Shot 3D Path Planning
HiWE is a hierarchical world knowledge model that enables zero‑shot 3D path planning by linking visual grounding with language‑based planning through a point‑based interface. It uses PointVLM to map task‑relevant objects to image coordinates, lifts these predictions into a semantic 3D representation with depth data, and then a language planner (3DLLM) generates end‑effector waypoints and gripper commands. The system is evaluated on 14 simulated manipulation tasks and four physical‑robot tasks, with ablations on visual training data, spatial inputs, and grasp selection.
arXiv Computer Vision
5d ago
ForeAct3D: Policy-Grounded Future World Modeling for VLA Policies
ForeAct3D is a framework that grounds future world modeling in Vision‑Language‑Action policies by decoding depth, semantic segmentation, and camera pose from policy representations into current and future 3D scene states. The future queries are conditioned on the policy‑generated action chunk, and a physical‑consistency closure enforces background staticity, instance‑level rigidity, and wrist‑camera pose alignment with end‑effector kinematics. This approach eliminates the need for future prediction at inference and achieves high success rates on LIBERO and CALVIN, outperforming the base policy across all suites.
arXiv AI
Jul 7
DSWAM: A Dual-System World Action Foundation Model for Fine-Grained Robot Manipulation
arXiv:2607. 04927v1 Announce Type: cross Abstract: World Action Models (WAMs) provide a promising alternative to Vision-Language-Action (VLA) policies by using video-based world modeling as dense supervision for robot action learning.
arXiv AI
Sep 10
CLAMP: Constrained Decoding for Vision-Language Embodied Planning
arXiv:2609.08602v1 Announce Type: new Abstract: Embodied planning increasingly relies on vision-language models (VLMs) to translate instructions and visual observations into executable action sequenc...