Hugging Face Trending Papers August 27, 2026

Decoupling Planning and Control for Instructable Agents

Read the original on Hugging Face Trending Papers →

The paper introduces Instruct-to-Act, a system that decouples high‑level planning from low‑latency control by combining a vision‑language model (VLM) planner with a world‑model controller. The VLM generates sparse, high‑level text instructions, while the controller executes them autonomously at high frequency. Experiments across seven embodied environments, including multi‑agent settings, show that this approach outperforms both controller‑only and direct VLM action‑generation methods, maintains fast control, and allows swapping in different pretrained VLM planners without fine‑tuning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI

Aug 28

Decoupling Planning and Control for Instructable Agents

The paper introduces Instruct-to-Act, a system that decouples planning and control by combining a vision‑language model (VLM) planner with a world‑model controller. The VLM generates sparse, high‑level text instructions, while the controller executes them at high frequency, trained via relabeling rollouts with synthetic instructions and joint optimization of behavior cloning, reward, and world‑model objectives. Across seven embodied environments—including multi‑agent settings—this approach outperforms controller‑only and direct VLM action methods, maintains fast control, and allows swapping pretrained VLM planners without fine‑tuning, achieving competitive results with strong baselines on most tasks.

By Zineng Tang, Kelsey R. Allen, Sjoerd van Steenkiste, Ishita Dasgupta, Alane Suhr

llms agents reinforcement-learning robotics fine-tuning multimodal

arXiv AI

Jun 10

RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning

arXiv:2510. 14828v3 Announce Type: replace Abstract: Improving the reasoning capabilities of embodied agents is crucial for robots to complete complex human instructions in long-view manipulation tasks successfully.

By Jinrui Liu, Bingyan Nie, Boyu Li, Yaran Chen, Yuze Wang, Shunsen He, Haoran Li

llms agents reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Aug 6

Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models

arXiv:2608. 04765v1 Announce Type: cross Abstract: Vision-language-action (VLA) models provide a unified paradigm for connecting visual perception, language understanding, and robotic control.

By Houze Xu, Jizhong Li, Ziyi Ye

llms nlp robotics fine-tuning multimodal

arXiv Machine Learning

Jun 10

Task Robustness via Re-Labelling Vision-Action Robot Data

arXiv:2606. 10918v1 Announce Type: cross Abstract: The recent trend in scaling models for robot learning has resulted in impressive policies that can perform various manipulation tasks and generalize to novel scenarios.

By Artur Kuramshin, \"Ozg\"ur Aslan, Cyrus Neary, Glen Berseth

llms robotics multimodal

arXiv AI

Jun 17

PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space

arXiv:2606. 17924v1 Announce Type: cross Abstract: Current Vision-Language-Action (VLA) models face a trade-off between efficient action generation and explicit deliberation.

By Bochen Yang, Lianlei Shan

llms diffusion reinforcement-learning robotics multimodal benchmarks

arXiv AI

Jul 7

Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulation

arXiv:2607. 05377v1 Announce Type: cross Abstract: While recent Vision-Language-Action (VLA) models show promise toward generalist manipulation policies, they struggle with long-horizon tasks due to their Markovian nature-relying solely on current observations.

By Jiaqi Peng, Xiqian Yu, Delin Feng, Yuqiang Yang, Wenzhe Cai, Jing Xiong, Ganlin Yang, Jinliang Zheng, Jiafei Cao, Xueyuan Wei, Jiangmiao Pang, Yuan Shen, Tai Wang

agents robotics fine-tuning multimodal

One email a morning, machine-written

Related stories

Decoupling Planning and Control for Instructable Agents

RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning

Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models

Task Robustness via Re-Labelling Vision-Action Robot Data

PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space

Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulation