arXiv AI By Yehang Zhang, Haojian Huang, Yifan Chang, Jianchong Su, Bohan Zhou, Yingjie Xu, Wosong Chen, Tianhao Zhou, Chenxu Wang, Tianyi Zhang, Yangkai Wei, Wenqian Li, Shiyuan Deng, Yinchuan Li, Ying-Cong Chen, Zexi Li

World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsal

Read the original on arXiv AI →

World Action Agent (WAA) is a multi‑agent framework that lets vision‑language models (VLMs) directly pilot robots by operating within a visual action workspace. The workspace provides automatically selected contact views, editable action rehearsals, and in‑view correction to refine decisions before low‑level execution. WAA learns procedural skills from expert videos and human teaching, and its interaction traces can train smaller VLMs, achieving state‑of‑the‑art success on LIBERO‑Pro and improving out‑of‑domain performance on robosuite and Qwen3.5‑9B.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Nov 9

On Agentic Manipulation Policies

The paper introduces Agentic Manipulation Policies (AMPs), where a generalist AI model directly controls a robot manipulator without task‑specific training or demonstrations. AMP1, a reference harness, achieves state‑of‑the‑art zero‑shot success rates on RoboLab‑120 and CaP‑Gym, outperforming specialist vision‑language‑action and world‑action models. The agent autonomously develops strategies such as searching for hidden objects and using tools, and adding a persistent memory further reduces failure rates on bimanual tasks.

By Chen Chen, Omid Bagherieh, Naveen Nikamanth Appiah Balaji, Jesse T. Gonzalez, Negin Heravi, Fahad Islam, Xueting Li, Jean-Baptiste Regli, Kejia Ren, Petr \v{S}vec, David J. Yoon, Matthias M\"uller, Vladlen Koltun
arXiv AI
Oct 27

Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy

The report evaluates the GPT-6-Astra multimodal large language model (MLLM) on the EBench platform, comparing two agent configurations: direct control, where the agent generates waypoint commands, and harness with policy, where it can invoke post‑trained vision‑language‑action (VLA) or world action model (WAM) policies. Direct control performs competitively overall, excelling on short‑horizon tasks and showing stable generalization, but falls behind on long‑horizon, tabletop, and high‑precision contact‑rich tasks. The harness with policy approach combines the strengths of both components, achieving success rates comparable to the best post‑trained policies while reducing token cost, yet still lags behind the standalone upper bounds due to suboptimal policy invocation and execution issues.

By Yifei Yao, Yude Zou, Guanbo Jiao, Xing Gao, Ning Gao, Yukai Wang, Jiantong Chen, Zihou Zhu, Xinyu Li, Ningyu Yan, Shuai Wang, Jianming Ma, Haoyu Guo, Weinan Zhang, Chunhua Shen