arXiv AI
Testing the Planning Claims of LLM-Based Agents against Formal Semantics
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Aug 28
INTENT-AS-A-TOOL Makes it Easy to Track Agentic Misalignment
The paper introduces INTENT-AS-A-TOOL, a method that equips large language models with intent-targeted tools to provide a fine-grained, judge‑free signal of their commitment to specific behaviors during reasoning. By monitoring the probability of calling these intent tools, the authors can track how intent evolves throughout generation, complementing chain‑of‑thought monitoring and expanding post‑hoc labels into dense trajectories. The approach identifies critical steps for online intervention, demonstrating that action preferences are useful for detecting agentic misalignment in autonomous agents.
Hugging Face Trending Papers
Jun 19
Training the Orchestrator: A Supervised Approach to End-to-End PDDL Planning with LLM Agents
Translating natural-language planning intent into verified plans is a longstanding challenge: people communicate goals in language, while classical planners require formal PDDL specifications. Recent agentic frameworks bridge this gap by orchestrating a pool of specialized repair agents inside a verifier-checked refinement loop, but the orchestrator at the centre is itself a prompted frontier LLM, paying a frontier-LLM API call at every refinement step.
arXiv AI
4d ago
Same Context, Different Agents: A Behavioral Study of Semantic Layers, Orchestration and Compaction in Data Agents
The study examines how three context‑engineering techniques—semantic layers, orchestration, and compaction—affect data agents that answer questions over heterogeneous data. Using seven large language models on 1,514 long‑horizon tasks, the authors identify three agent behavior groups (investigators, premature answerers, early stoppers) and show that each technique benefits only the group it aligns with, while sometimes harming others. Combining the techniques yields smaller net gains than the sum of individual effects, leading to guidance on selecting techniques based on agent behavior, task demands, and accuracy‑cost trade‑offs.
arXiv AI
Aug 10
From Plan to Action: How Well Do Agents Follow the Plan?
arXiv:2604. 12147v3 Announce Type: replace-cross Abstract: Agents are commonly instructed to follow a task-specific plan for guidance.
Hugging Face Trending Papers
Jun 30
Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action
Theory of Mind (ToM) benchmarks for Large Language Models (LLMs) typically rely on passive question-answering formats, but the deployment of LLMs in increasingly agentic and autonomous forms demands new evaluations. In this paper we evaluate an agent's ability to induce specific belief states in other agents by taking actions rather than using conversational persuasion, a capability we call Non-Conversational Planning ToM (NCP-ToM).
arXiv AI
Jul 8
Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
arXiv:2607. 05775v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly evaluated on their ability to use tools, plan multi-step tasks, coordinate with other agents, and operate over extended horizons.