The paper introduces a closed‑loop framework to evaluate frontier AI models on the LIBERO‑10 robot manipulation benchmark, allowing models to control a robot via visual inputs, proprioception, and tool calls without task‑specific fine‑tuning. Six models (GPT‑6 Astra, GPT‑6.1 Sol, GPT‑6 Luna, Claude Opus 5, Claude Opus 5.5, Gemini‑4 Argon) were tested across 18 configurations and 1,800 episodes, with GPT‑6 Astra achieving the highest success rate of 44%. Results show that higher reasoning effort generally improves performance, yet benefits vary by model and task, and many models struggle with precise spatial arrangements and complex multi‑step interactions.
Recent advances in frontier AI models have demonstrated remarkable capabilities in visual understanding, reasoning, planning, and tool use, raising the possibility of using general-purpose AI models f...
arXiv:2607. 15641v1 Announce Type: cross Abstract: Humans combine reasoning and motor control to solve complex manipulation tasks under diverse constraints.
arXiv:2510. 14828v3 Announce Type: replace Abstract: Improving the reasoning capabilities of embodied agents is crucial for robots to complete complex human instructions in long-view manipulation tasks successfully.
AffordSim is a scalable data generator and benchmark that integrates open‑vocabulary 3D affordance prediction into simulation‑based trajectory generation for robotic manipulation. Given a natural‑language task description, it synthesizes task‑relevant scenes, queries affordances, grounds them on object surfaces, samples region‑conditioned grasps, and selects executable candidates with motion planning, while randomizing pose, texture, lighting, noise, and backgrounds for sim‑to‑real transfer. The benchmark includes 50 tasks, five robot embodiments, and over 500 objects, achieving 93% of the success rate of manual contact annotations on affordance‑critical tasks and 89% on hard composite tasks, with policies trained on AffordSim transferring zero‑shot to a real Franka FR3 with 24% average success.
arXiv:2606. 08530v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models achieve strong benchmark performance but still struggle in real-world deployment with unseen objects, background shifts, and different robot embodiments.
arXiv:2610.10409v1 Announce Type: cross Abstract: General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities car...
arXiv:2512. 24125v3 Announce Type: replace-cross Abstract: General-purpose robotic systems operating in open-world environments must achieve both broad generalization and high-precision action execution, a combination that remains challenging for existing Vision-Language-Action (VLA) models.
arXiv:2607. 14341v1 Announce Type: cross Abstract: Robust robotic grasping remains a fundamental challenge for complex real-world applications.
arXiv:2609.12541v1 Announce Type: new Abstract: We demonstrate that a general-purpose agent can directly drive a physical robot throughout task execution without any task-specific or environment-spec...
arXiv:2610.36533v1 Announce Type: cross Abstract: Recently, LLM-based autonomous agents have shown strong capabilities in iteratively generating, evaluating, and refining executable solutions. In rob...
arXiv:2606.18363v3 Announce Type: replace-cross Abstract: Language models trained on large-scale vision-language data have demonstrated strong potential for embodied agents. Harnessing models through...
RoboSPA is a large-scale robotic manipulation dataset and benchmark designed to evaluate Vision‑Language‑Action models on fine‑grained spatial reasoning and long‑horizon procedural planning. It contains 10 task categories, 56 base tasks, and 280 variants across five difficulty levels, with 527K trajectories collected from multiple embodiments and scenes. The benchmark introduces diagnostic metrics beyond binary success, revealing that current VLA models struggle with complex spatial relations, precise execution, and memory‑intensive planning.