Let the Mind Conceive, Let the Hand Achieve: Following Multimodal Instructions for Long-Horizon Robotic Manipulation

The paper introduces Orchestra, a robotic agent that separates task planning from action generation to follow multimodal, long‑horizon instructions. Orchestra decomposes instructions into ordered subtasks, tracks progress, updates object locations, and uses a progress evaluator to decide when to advance. On the new RMBench‑Compose benchmark, Orchestra outperforms prior methods, achieving higher success rates and longer average subtask sequences, and shows improved performance on real‑world multi‑step tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• agents
• robotics
• multimodal
• benchmarks