arXiv AI
How Far Are LLMs from Symbolic Planners? An NLP-Based Perspective
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 18
PDDLCoder: Agentic PDDL Generation for LLM-Assisted Symbolic Planning
arXiv:2608. 16637v1 Announce Type: new Abstract: LLMs remain unreliable for long-horizon planning, often generating logically inconsistent or non-applicable plans.
arXiv AI
6d ago
Testing the Planning Claims of LLM-Based Agents against Formal Semantics
The paper evaluates the planning abilities of large language model (LLM) agents by comparing their performance against formal semantics defined in hidden PDDL models. Across five domains, four LLMs, six prompting strategies, and three agent architectures, the best setup solved 78% of solvable tasks offline but only 40% online, with limited recognition of unsolvable tasks and no clear benefit from more elaborate prompting. The study concludes that claims of LLM planning competence require validated action traces rather than end‑to‑end success rates.
arXiv AI
Jun 29
Towards Reliable and Robust LLM Planning: Symbolic Feedback-Driven Iterative Self-Refinement Framework
arXiv:2606. 27757v1 Announce Type: new Abstract: Large language models (LLMs) have attracted widespread attention from academia and industry, yet their deployment raises critical security concerns regarding robustness and reliability.
arXiv Computation and Language
2d ago
MemPlan: Memory-Conditioned PDDL Planning for Partially Observable Text Environments
arXiv:2610.30379v1 Announce Type: new Abstract: LLM agents are flexible in language interaction, but using them as long-horizon planners is costly and prone to hallucinated or invalid actions. This p...
arXiv Computation and Language
2d ago
Language Models as Higher-Order Planning Formalizers
The paper examines the limitations of using large language models (LLMs) to translate natural language planning problems into PDDL representations, noting that standard benchmarks often map directly to PDDL and may overstate success. It introduces the concept of "unraveling problems," where concise natural language descriptions correspond to large PDDL structures, and shows that traditional LLM formalizers struggle to scale on such variants. To address this, the authors propose a new paradigm, LLM-as-Higher-Order-Formalizer, where the LLM first generates a high‑level program that encapsulates recurrent logic, which then produces the full PDDL representation, thereby decoupling token output from combinatorial explosion and improving performance on complex tasks.
arXiv AI
Jun 3
A Close Look At World Model Recovery In Supervised Fine-Tuned LLM Planners
arXiv:2606. 03685v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) improves end-to-end classical planning in large language models (LLMs), but do these models also learn to represent and reason about the planning problems they are solving?