UniRAP is a unified model that infers part-level physical affordances and translates language instructions into actionable geometric representations. It treats the task as a conditional multimodal generation problem, integrating visual inputs, textual instructions, and optional prompts into a shared spatiotemporal representation via a unified interface token mechanism. The model employs a Unified Affordance Decoder to jointly perform object detection, part-level affordance segmentation, and 4‑DoF interaction pose estimation, and uses a curriculum-based transfer training strategy to improve data efficiency with limited textual annotations.
Vision-language perception has achieved impressive progress in aligning natural language with visual observations, yet grounding high-level semantics into part-level physical interaction remains chall...
arXiv:2609.37264v1 Announce Type: cross Abstract: Affordance perception aims to localize actionable regions supporting embodied interaction, yet 2D and 3D affordance grounding have evolved as separat...
arXiv:2607. 03595v1 Announce Type: cross Abstract: Affordance grounding aims to localize image regions that support a specific action, serving as a core capability for physical intelligence and embodied perception.
AffordSim is a scalable data generator and benchmark that integrates open‑vocabulary 3D affordance prediction into simulation‑based trajectory generation for robotic manipulation. Given a natural‑language task description, it synthesizes task‑relevant scenes, queries affordances, grounds them on object surfaces, samples region‑conditioned grasps, and selects executable candidates with motion planning, while randomizing pose, texture, lighting, noise, and backgrounds for sim‑to‑real transfer. The benchmark includes 50 tasks, five robot embodiments, and over 500 objects, achieving 93% of the success rate of manual contact annotations on affordance‑critical tasks and 89% on hard composite tasks, with policies trained on AffordSim transferring zero‑shot to a real Franka FR3 with 24% average success.
The paper introduces a metric interaction framework for robotic manipulation that explicitly models object- and scene-level interactions in Cartesian space. It uses Interaction‑Centric Tokens (ICTs) to represent end‑effector trajectories relative to objects and a Metric Action Interaction Field (MAIF) to attend to scene point‑cloud features for geometry‑conditioned action corrections. Experiments show modest but consistent improvements across several benchmarks, including LIBERO, RoboTwin 2.0, and real‑world tasks.
arXiv:2609.22332v1 Announce Type: cross Abstract: Generalizable robot manipulation requires predicting how a scene will evolve, identifying where interactions are feasible, and determining how to act...
arXiv:2605.03927v3 Announce Type: replace Abstract: Vision-language models have demonstrated strong performance across robotic perception and instruction-following tasks. However, they still struggle...
arXiv:2609.38466v1 Announce Type: new Abstract: Text-conditioned full-body human-object interaction (HOI) generation requires synthesizing human motion and object trajectories that match the input te...
arXiv:2610.09718v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models acquire broad manipulation capabilities via large-scale pretraining, yet eliciting them through language requires...
arXiv:2512.04231v2 Announce Type: replace-cross Abstract: Assistive robots operating in unstructured environments must reason not only about what objects are, but how they can be used, including non-...
Vision-Language-Action models and World-Action Models have advanced language-conditioned robotic manipulation, yet often leave metric relations among actions, objects, and scene geometry implicit. Hum...
PhysBrain 1.5 is a unified vision‑language model that learns to understand physical environments, generate actions, and predict future states by encoding language, end‑effector motion, and dense visual targets as discrete sequences and training them with autoregressive next‑token prediction. The model is pre‑trained on human interaction videos and fine‑tuned on human demonstrations, robot trajectories, and simulated experience, achieving an average score of 72.5 across 28 embodied understanding benchmarks and outperforming other open‑source models on 14 of them. It also demonstrates the ability to produce end‑effector trajectories and predict future scenes with spatially aligned RGB, depth, and robot‑mask outputs.