arXiv AI
Jun 26
WatchAct: A Benchmark for Behavior-Grounded Robot Manipulation
arXiv:2606. 26443v1 Announce Type: cross Abstract: A robot working alongside people must reason about what they have done, in what order, and with what intent.
Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policies
Related stories
arXiv Machine Learning
Oct 8
RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments
arXiv:2610.10409v1 Announce Type: cross Abstract: General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities car...
arXiv AI
Jul 22
Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents
arXiv:2607. 19190v1 Announce Type: cross Abstract: Real-to-sim conversion for robotic interaction with objects remains labor-intensive because it requires more than visual reconstruction: a streamlined real2sim process must recover scene geometries and object states, infer physical parameters, and assemble actors, objects, cameras, poses, and trajectories into a runnable physical simulation.
arXiv Computer Vision
Sep 30
Real2Gym: Building Gyms from Videos, Bringing Skills to Robots
arXiv:2609.37089v1 Announce Type: new Abstract: Real-world videos provide rich demonstrations of manipulation, but turning them into reusable robot skills requires visually aligned environments, exec...
arXiv AI
1d ago
D-World: An Explicit World-Language-Action Model with Recursive Self-Improvement under Dual Control
arXiv:2610.17114v1 Announce Type: cross Abstract: Frontier large language models (LLMs) now reason about space well enough to program robots, yet the world models behind robot policies are implicit,...
arXiv Computer Vision
Oct 8
Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
arXiv:2610.04432v2 Announce Type: replace Abstract: Building interactive simulators from real-world observations is a promising way to scale embodied data, but current pipelines still rely heavily on...
arXiv AI
Jul 14
VIA: Visual Interface Agent for Robot Control
arXiv:2607. 11119v1 Announce Type: cross Abstract: Robot manipulation is a complex task that requires visual understanding, physical reasoning, planning, and closed-loop control.
Hugging Face Trending Papers
Oct 7
RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments
General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carry into the physical world. To investigate this, we...
Hugging Face Trending Papers
Sep 17
Learning and Transferring Closed-Loop Robot Software
Closed‑loop robot policies are difficult to design manually because they require complex observation processing, state management, and branching. This study treats complete closed‑loop implementations as reusable execution experience: a coding agent generates policy code from a few demonstrations, iteratively improves it with simulation feedback, and stores the validated implementations. When applying these archived implementations to new tasks, the agent can generate and refine policies using the stored code, target demonstrations, and execution feedback, ultimately producing a frozen policy that runs without further model calls. Across multiple source and target tasks, iterative optimization of the source implementations significantly boosts success rates, demonstrating the value of execution‑improved software for acquiring new policies.
arXiv AI
Oct 2
Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents
Reconstruct, Practice, Go Real (RPG) is a framework that enables autonomous improvement of robot execution systems without updating model weights. RPG identifies manipulation capabilities from offline data, creates simulation practice tasks, and uses feedback to diagnose failures, develop new symbolic skills, refine existing ones, and revise the system prompt. Across 22 manipulation tasks, RPG raises task success from 28.6% after the first practice round to 95.0% after 15 rounds, outperforming baselines and achieving perfect success on 30 physical trials after calibration.
arXiv Machine Learning
4d ago
Fly-by-Code: Embodied Coding Agents for Aerial Manipulation with Active Visual and Physical Feedback
Fly-by-Code introduces embodied coding agents that generate robot policies from natural-language instructions and refine them using execution feedback. In aerial manipulation, partial observability and fragile interactions limit the usefulness of standard feedback, so the authors propose Active Feedback (AF), a framework that synthesizes diagnostic programs to view unseen outcomes or probe failure causes. Experiments on six simulated tasks show AF dramatically reduces false completions and boosts task success, and the refined programs transfer to a real aerial manipulator without code changes.
arXiv AI
Sep 25
World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsal
World Action Agent (WAA) is a multi‑agent framework that lets vision‑language models (VLMs) directly pilot robots by operating within a visual action workspace. The workspace provides automatically selected contact views, editable action rehearsals, and in‑view correction to refine decisions before low‑level execution. WAA learns procedural skills from expert videos and human teaching, and its interaction traces can train smaller VLMs, achieving state‑of‑the‑art success on LIBERO‑Pro and improving out‑of‑domain performance on robosuite and Qwen3.5‑9B.