arXiv AI
Jun 30
GROW$^2$: Grounding Which and Where for Robot Tool Use
arXiv:2606. 30632v1 Announce Type: cross Abstract: Can the robot use a plate to cut a cake if no knife is available?
VLMs Can Describe, But Not Measure: Object-Centric Scene Understanding for Robotic Manipulation
The paper introduces a modular perception framework that uses vision‑language models (VLMs) to annotate object‑level regions from a single RGB‑D observation, then grounds these annotations with depth data to build an object‑centric representation. Experiments on 151 tabletop scenes demonstrate that this decomposition maintains strong semantic performance while significantly improving localization and depth estimation compared to direct VLM inference. The resulting representation is integrated into a task‑planning system for robotic manipulation.
Related stories
arXiv AI
Jun 8
AxisGuide: Grounding Robot Action Coordinate System in RGB Observations for Robust Visuomotor Manipulation
arXiv:2606. 06761v1 Announce Type: cross Abstract: Visuomotor manipulation policies trained via large-scale behavior cloning have achieved strong semantic scene understanding, yet often fail to reliably execute correct low-level actions under distribution shifts.
arXiv AI
4d ago
Gondola: Grounded Vision Language Planning for Robotic Manipulation
arXiv:2506.11261v2 Announce Type: replace-cross Abstract: Vision-language-action (VLA) models have shown promising progress in robotic manipulation. However, directly mapping visual observations and...
arXiv Computer Vision
Aug 21
PVRA: A Pointwise Key-point Voting Framework for Robotic Assembly
arXiv:2608. 19968v1 Announce Type: cross Abstract: Modern computer vision has enabled partial autonomy in robotic assembly manipulation.
arXiv Computer Vision
Aug 27
V-Link: Recovering Lost Visual Representations in Action DiT for Vision-Language-Action Models
V-Link is a method designed to enhance Vision‑Language‑Action (VLA) models by recovering visual representations during the transfer from vision‑language (VL) features to action (A) features. It introduces complementary Spatial and Semantic Query representations that are injected into Action DiT through asymmetric pathways, providing both semantic augmentation and dedicated geometric conditioning for action generation. Experiments on LIBERO, LIBERO‑Plus, RoboTwin 2.0, and real‑world AGIBOT A3 Ultra tasks show significant performance gains over the base GR00T N1.6 model.
arXiv Computer Vision
Sep 10
StateVLM: A State-Aware Vision-Language Model for Robotic Affordance Reasoning
arXiv:2605.03927v3 Announce Type: replace Abstract: Vision-language models have demonstrated strong performance across robotic perception and instruction-following tasks. However, they still struggle...
arXiv Machine Learning
Jun 10
Task Robustness via Re-Labelling Vision-Action Robot Data
arXiv:2606. 10918v1 Announce Type: cross Abstract: The recent trend in scaling models for robot learning has resulted in impressive policies that can perform various manipulation tasks and generalize to novel scenarios.
arXiv AI
Aug 25
PEAfowl: Perception-Enhanced Multi-View Vision-Language-Action for Bimanual Manipulation
arXiv:2601.17885v2 Announce Type: replace-cross Abstract: Bimanual manipulation in cluttered scenes requires policies that remain stable under occlusions, viewpoint changes and scene variations. Exis...
arXiv AI
Sep 23
See like a Robot: Robot-Centric Pointmaps for VLA Models
arXiv:2607.11498v2 Announce Type: replace-cross Abstract: Vision-language-action (VLA) models require 3D spatial reasoning, yet RGB observations encode robot-object geometry only implicitly. Lifting...
arXiv Computer Vision
Sep 1
Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
arXiv:2508.13073v3 Announce Type: replace-cross Abstract: Robotic manipulation, a key frontier in robotics and embodied AI, requires precise motor control and multimodal understanding, yet traditiona...
Hugging Face Trending Papers
Sep 8
3DWay: Generalizing Robot Manipulation via 3D Consistent Waypoints
The paper introduces 3DWay, a method that predicts 3D consistent waypoints for robot manipulation using multi‑view images. By first generating 2D waypoints that are consistent across views and then triangulating them, the approach provides explicit 3D motion specifications while leveraging pretrained vision‑language models. Experiments demonstrate that 3DWay improves 3D spatial grounding and vision‑language reasoning, enhancing the generalization of robot manipulation policies.
arXiv AI
Jul 13
MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs
arXiv:2607. 08970v1 Announce Type: cross Abstract: Recent benchmarks for VLMs largely assess single- or limited-view perception, leaving untested the core cognitive ability to integrate observations across viewpoints into a coherent, world-centric (allocentric) 3D mental model.