Attacca is a new approach for training visual goal‑conditioned policies that can handle continuous long‑horizon task execution in embodied agents. It decouples goal images from the execution environment by sampling class‑compatible masked goals from other worlds, enabling the policy to ground targets in the current view via a target‑mask prediction head. The method also introduces behavioral‑phase conditioning to distinguish Search, Approach, and Interact stages, and achieves significant performance gains on Minecraft tasks, outperforming baselines by up to 7× on long‑horizon scenarios.
By Gyusik Seo, Jaehong Yoon
arXiv:2607. 17760v1 Announce Type: cross Abstract: Inverse reinforcement learning (IRL) provides a powerful framework for learning from demonstrations.
By Ziyi Liu, Grace Zhang
arXiv:2608.22301v1 Announce Type: cross
Abstract: Humans imitate at the level of intent: given a demonstration, we infer its goal and carry it out with whatever tools, objects, and layouts are at han...
By Xunzhe Zhou, Yiyang Cai, Fengyi Wang, Ran Ju, Hanxiang Ren, Ruizhe Liu, Yu Zhang, Qian Luo, Feng Chen, Pei Zhou, Yi Ma, Yanchao Yang
The paper introduces Instruct-to-Act, a system that decouples high‑level planning from low‑latency control by combining a vision‑language model (VLM) planner with a world‑model controller. The VLM generates sparse, high‑level text instructions, while the controller executes them autonomously at high frequency. Experiments across seven embodied environments, including multi‑agent settings, show that this approach outperforms both controller‑only and direct VLM action‑generation methods, maintains fast control, and allows swapping in different pretrained VLM planners without fine‑tuning.
arXiv:2608. 06756v1 Announce Type: new Abstract: Vision-language models are increasingly serving as the reasoning core of embodied agents.
By Ying Chen, Weizhen Li, Zhe Hu, Zhenjiang Li, Rui Jiang, Zhifeng Gu, Lihuang Fang, Jiangping Liu, Lei Yi, Jie Chen
The paper introduces Instruct-to-Act, a system that decouples planning and control by combining a vision‑language model (VLM) planner with a world‑model controller. The VLM generates sparse, high‑level text instructions, while the controller executes them at high frequency, trained via relabeling rollouts with synthetic instructions and joint optimization of behavior cloning, reward, and world‑model objectives. Across seven embodied environments—including multi‑agent settings—this approach outperforms controller‑only and direct VLM action methods, maintains fast control, and allows swapping pretrained VLM planners without fine‑tuning, achieving competitive results with strong baselines on most tasks.
By Zineng Tang, Kelsey R. Allen, Sjoerd van Steenkiste, Ishita Dasgupta, Alane Suhr
arXiv:2605. 30280v2 Announce Type: replace-cross Abstract: Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments.
By Qiuyue Wang, Mingsheng Li, Jian Guan, Jinhui Ye, Sicheng Xie, Yitao Liu, Junhao Chen, Zhixuan Liang, Jie Zhang, Xintong Hu, Xuhong Huang, Pei Lin, Junyang Lin, Dayiheng Liu, Shuai Bai, Jingren Zhou, Jiazhao Zhang, Haoqi Yuan, Gengze Zhou, Hang Yin, Ye Wang, Yiyang Huang, Zixing Lei, Wujian Peng, Delin Chen, Yingming Zheng, Jingyang Fan, Xianwei Zhuang, Xin Zhou, Haoyang Li, Anzhe Chen, Tong Zhang, Xuejing Liu, Yuchong Sun, Ruizhe Chen, Zhaohai Li, Chenxu L\"u, Zhibo Yang, Tao Yu, Xionghui Chen
arXiv:2609.39915v1 Announce Type: new
Abstract: Vision-Language Navigation (VLN) requires embodied agents to generate actions based on instructions and observations. General-purpose multimodal agents...
By Haoxiang Shi, Zaijing Li, Muhe Ding, Xiang Deng, Yaowei Wang, Liqiang Nie
WorldGuide is a closed‑loop video world model that, given an initial image and a task goal, predicts an atomic action, generates the corresponding video clip, and uses that clip to decide the next action or terminate the task. It trains a Planner to predict the next action or completion from visual progress and an Executor to realize those actions, while a hierarchical visual memory tracks state over long horizons. The authors introduce WorldGuide‑Bench, a dataset of ~59K step‑annotated videos across 245 tasks, and show that WorldGuide outperforms the recent MiniMax‑H3 model on both WorldGuide‑Bench and VideoCraft‑Bench, highlighting the benefit of coupling planning with learned execution for goal‑directed procedural video generation.
By Ankan Deria, Komal Kumar, Hisham Cholakkal, Fahad Shahbaz Khan, Salman Khan
arXiv:2605. 14211v3 Announce Type: replace Abstract: Long-horizon embodied tasks remain a fundamental challenge in AI, as current methods rely on hand-engineered rewards or action-labeled demonstrations, neither of which scales.
By Benjamin Schneider, Xavier Schneider, Victor Zhong, Sun Sun
Inverse reinforcement learning (IRL) provides a powerful framework for learning from demonstrations. However, real-world tasks often exhibit substantial natural variations (e.
arXiv:2606.20627v2 Announce Type: replace
Abstract: Joint-Embedding Predictive Architectures (JEPAs) enable agents to plan in latent space by imagining the outcomes of candidate actions, yet task spe...
By Samuel Barbeau, Simon Roy, Giovanni Beltrame, Christian Desrosiers, Nicolas Thome