arXiv AI
Jun 30
GROW$^2$: Grounding Which and Where for Robot Tool Use
arXiv:2606. 30632v1 Announce Type: cross Abstract: Can the robot use a plate to cut a cake if no knife is available?
StateVLM: A State-Aware Vision-Language Model for Robotic Affordance Reasoning
Related stories
arXiv Computer Vision
Sep 23
Metric-Bench: Exploring In-context Spatial Metric Reasoning in VLMs for Indoor Scenes
Metric-Bench introduces a new benchmark for Vision‑Language Models (VLMs) that focuses on metric‑spatial reasoning in indoor scenes by using in‑image reference objects with known dimensions. The accompanying MetricReasoner fine‑tuning recipe employs structured prompts and numerical rewards to implicitly learn 2D‑to‑3D mapping without camera intrinsics. Experiments show that this approach improves spatial metric understanding by 43.1 % over existing models and boosts downstream embodied tasks, while also delivering gains on general VLM benchmarks.
arXiv AI
Jul 7
Token-Based Affordance Grounding with Large Vision-Language Models
arXiv:2607. 03595v1 Announce Type: cross Abstract: Affordance grounding aims to localize image regions that support a specific action, serving as a core capability for physical intelligence and embodied perception.
arXiv Computer Vision
Aug 24
AffordAny: Open-World 3D Affordance Grounding from Monocular RGB Images via Vision-Language-Guided Geometric Reasoning
arXiv:2608.20720v1 Announce Type: new Abstract: Open-world 3D affordance grounding requires localizing functional object parts in 3D given free-form language queries. Existing methods typically assum...
arXiv AI
2d ago
CRAFT-E: An Interpretable and Editable Neuro-Symbolic Framework for Affordance-Grounded Active Object Retrieval
arXiv:2512.04231v2 Announce Type: replace-cross Abstract: Assistive robots operating in unstructured environments must reason not only about what objects are, but how they can be used, including non-...
arXiv AI
Aug 13
SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward
arXiv:2608. 12220v1 Announce Type: cross Abstract: Existing Vision-Language Models (VLMs) exhibits a critical bottleneck in robust spatial reasoning.
arXiv AI
Jun 17
Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
arXiv:2606. 17539v1 Announce Type: cross Abstract: Spatial VLMs have made substantial progress in geometric perception, yet complex spatial reasoning requiring multi-step inference over depth, distance, and scene relations remains challenging.
arXiv Machine Learning
Oct 19
Look, Reason, Draw a Region: Recursive Refinement for Spatial Context Prediction
arXiv:2610.23302v1 Announce Type: cross Abstract: Spatial context prediction requires visual grounding and spatial reasoning to identify vacant regions relative to objects in a scene. Spatial relatio...
arXiv AI
Sep 2
Towards Generalizable Visually Grounded Exploration of Household Devices
The paper introduces VGEBench, a new benchmark for evaluating Vision‑Language Models (VLMs) on generalizable, visually grounded exploration of household devices. Unlike existing datasets that rely on static images or annotated trajectories, VGEBench employs a logic‑driven state machine to simulate multi‑turn interaction loops, requiring agents to actively perceive, act, and refine their actions to achieve goals. Experiments show that current VLMs struggle to translate semantic knowledge into physical execution and to maintain long‑horizon state tracking.
arXiv Computer Vision
Oct 16
DK-VLM: Geometry as Position for Spatial Reasoning in Vision-Language Models
DK‑VLM introduces Dual‑Kernel Attention, where a visual token’s attention is computed from both its input position and its geometric position derived from 3D points and camera poses. A semantic kernel preserves pretrained attention while a geometric kernel modulates attention via point displacements and relative view transformations, with a learned gate fusing the two. The model also employs Reference‑Frame Composition supervision to update its reference frame across successive judgments, achieving top scores on six spatial benchmarks and strong performance on VSI‑Bench and ReVSI‑32.
arXiv Machine Learning
Jun 5
What Objects Enable, Not What They Are: Functional Latent Spaces for Affordance Reasoning
arXiv:2606. 05533v1 Announce Type: new Abstract: Existing robot planning systems rely on appearance-based reasoning, where visual observations are encoded into latent spaces organized around object appearances (e.
arXiv AI
2d ago
AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulation
AffordSim is a scalable data generator and benchmark that integrates open‑vocabulary 3D affordance prediction into simulation‑based trajectory generation for robotic manipulation. Given a natural‑language task description, it synthesizes task‑relevant scenes, queries affordances, grounds them on object surfaces, samples region‑conditioned grasps, and selects executable candidates with motion planning, while randomizing pose, texture, lighting, noise, and backgrounds for sim‑to‑real transfer. The benchmark includes 50 tasks, five robot embodiments, and over 500 objects, achieving 93% of the success rate of manual contact annotations on affordance‑critical tasks and 89% on hard composite tasks, with policies trained on AffordSim transferring zero‑shot to a real Franka FR3 with 24% average success.