arXiv AI

Verifiable Efficiency Limits of Innate LLM Planning in Sliding Puzzles

The paper introduces a verifiable framework for measuring the innate planning efficiency of large language models (LLMs) on sliding puzzles, focusing on complete plans generated without external tools. It defines a Success-Weighted Plan Efficiency (SPE) metric as the product of execution reliability, planning coordination, and action economy, and validates these factors through trajectory replay and certified optimality. Experiments on 4×4 puzzles show that models can finish all trials yet become less economical over longer horizons, and that high efficiency among completed plans can coexist with poor goal attainment, highlighting distinct coordination and completion issues.

arXiv AI
Sep 25

GRASP: Generating, Revising, and Assessing for Strategic Planning with Agentic AI

GRASP is a multi-stage planning framework that improves the reliability of large language models on complex tasks. It separates planning into three specialized modules—GenPlan for global macro-guidelines, RevPlan for exploring localized strategies, and VerPlan for multi-criteria evaluation—allowing context isolation and strict macro-regularization. Experiments show GRASP outperforms direct LLM planners by significant margins on datasets such as Natural Plan Calendar Scheduling, ZebraLogic, and SciBench Math, and it mitigates performance collapse in multi-task and dual-task settings.

By Arunabh Srivastava (Amir), Mohammad A. (Amir), Khojastepour, Srimat Chakradhar, Sennur Ulukus
Hugging Face Trending Papers
Sep 24

GRASP: Generating, Revising, and Assessing for Strategic Planning with Agentic AI

GRASP is a multi-stage planning framework that separates planning into specialized modules: GenPlan for global macro-guidelines, RevPlan for exploring localized strategies, and VerPlan for multi-criteria evaluation. This strategy-aware approach yields state‑of‑the‑art accuracy on diverse datasets, outperforming direct LLM planners by up to 30.8% on ZebraLogic and reducing multi‑task degradation. GRASP’s context isolation and macro‑regularization also give it a 14.5% edge over frontier reasoning models like GPT‑5‑mini.

arXiv AI
Sep 3

PIE-APT: Abductive Planning over Temporal Dynamic Knowledge Graphs via Incremental Reasoning

PIE-APT introduces a unified framework for abductive planning over Temporal Dynamic Knowledge Graphs (TDKGs) using two modules: PIE-Abducer, which performs incremental direct-derivation abduction, and PIE-APT, which interleaves backward‑chaining A* search with PIE-Abducer to generate action sequences and abductive assumptions. The approach operates natively on the expressive SROIQ Description Logic, leveraging an incremental reasoner to maintain decidability and bypass the Ramification Problem. Evaluation on four OWL benchmarks demonstrates qualitative superiority over classical planners and shows that the direct‑derivation method outperforms a Minimal Hitting Set baseline in abductive enrichment.

By Amir Hossein Sharafi, Alireza Shahbazi
arXiv Computation and Language
Oct 28

Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agents

The paper introduces the Agent Planning Benchmark (APB), a diagnostic framework designed to evaluate the planning capabilities of large language model (LLM) agents. APB contains 4,209 multimodal cases across 22 domains and five settings, testing holistic planning, step-wise planning with feedback, and robustness to extraneous or broken tools and unsolvable tasks. Evaluation on 12 MLLMs reveals systematic weaknesses in long-horizon planning, tool-noise robustness, calibrated refusal, and inference-time refinement, while APB-guided refinement improves plan correctness and downstream execution metrics on ToolSandbox and τ²‑bench tasks.

By Haoyu Sun, Wenxuan Wang, Mingyang Song, Jujie He, Weinan Zhang, Yang Liu, Yang Yang, Yu Cheng
arXiv Computation and Language
Sep 14

LLM-BabyBench: Can Language Models Plan in Worlds They Can Simulate?

LLM‑BabyBench transforms the BabyAI gridworld into a fully observable, purely textual setting that isolates planning as the sole source of failure. By serialising the entire grid, providing formal instructions, and validating actions deterministically, the benchmark introduces the PPD suite—Predict, Plan, and Decompose tasks—each scored with metrics that separate mission understanding from sequencing. Across a range of large language models, simulation accuracy is high while planning success drops sharply beyond a model‑specific horizon, revealing that plan length—not grid size—drives failure and that models often commit to a single corridor‑shaped route without backtracking.

By Idriss Malek, Omar Choukrani, Daniil Orel, Anh Duy Le Dinh, Zhuohan Xie, Zangir Iklassov, Martin Tak\'a\v{c}, Salem Lahlou