arXiv Computation and Language
Sep 14

LLM-BabyBench: Can Language Models Plan in Worlds They Can Simulate?

LLM‑BabyBench transforms the BabyAI gridworld into a fully observable, purely textual setting that isolates planning as the sole source of failure. By serialising the entire grid, providing formal instructions, and validating actions deterministically, the benchmark introduces the PPD suite—Predict, Plan, and Decompose tasks—each scored with metrics that separate mission understanding from sequencing. Across a range of large language models, simulation accuracy is high while planning success drops sharply beyond a model‑specific horizon, revealing that plan length—not grid size—drives failure and that models often commit to a single corridor‑shaped route without backtracking.

By Idriss Malek, Omar Choukrani, Daniil Orel, Anh Duy Le Dinh, Zhuohan Xie, Zangir Iklassov, Martin Tak\'a\v{c}, Salem Lahlou
Hugging Face Trending Papers
Jul 9

LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Action

Vision-language-action (VLA) models aim to map multimodal inputs to robot actions. However, most existing approaches struggle to cover complex dynamic scenarios due to treating all visual tokens uniformly and reasoning with human-selected factors, which lack mechanisms to emphasize task-critical evidence and ignore underlying factors.

arXiv AI
Aug 25

Meta-Ctrl: Guaranteed Plan Generation by Decoupling Syntactic and Semantic Constraints

Meta-Ctrl is a constrained‑decoding framework that guarantees syntactic and semantic constraints in robot plans generated by large language models. It introduces meta‑tokens—a compact vocabulary of grounded actions—to enforce syntax at the token level and semantics (preconditions, goals, ordering) at the action level, dramatically reducing memory usage from over 107 TB to under 2 GB. The approach enables a small open‑weight LM to outperform GPT‑4 on subgoal success rates in the WAH‑NL LoTa‑Bench protocol and consistently improves performance on the Embodied Agent Interface, while real‑world tabletop robot experiments show every plan satisfies its preconditions and goals by construction.

By Gwen Yidou-Weng, Edward Sun, Tianyi Ma, Metin Alp Dogan, Benjie Wang, Allen Peng, Guy Van den Broeck, Yuchen Cui