The paper introduces the Belief-State Engine (BSE), an inference module that supplies a large language model (LLM) with a Bayesian posterior over hidden states in a partially observable Markov decision process (POMDP). By keeping the raw action‑observation log hidden from the LLM, the BSE ensures the agent behaves as a sound Markov policy on the belief MDP, thereby inheriting classical POMDP optimality guarantees. Experiments on the Tiger POMDP and a red‑team attack‑graph task show that BSE‑augmented agents outperform six baselines in task return, belief calibration, and decision consistency.
arXiv:2607. 28942v1 Announce Type: new Abstract: Recently Large Language Models (LLMs) have been increasingly deployed as autonomous agents in applications such as self-reflection, retrieval-augmented generation, and scientific discovery.
arXiv:2609.00455v1 Announce Type: new Abstract: Large language models (LLMs) are being used as policies for autonomous decision-making and planning in many domains. Despite their strong reasoning cap...
arXiv:2606. 19559v1 Announce Type: new Abstract: Recent position papers argue that the classical aleatoric/epistemic uncertainty framework is insufficient for interactive large language model (LLM) agents and call for underspecification-aware, decomposed, and communicable uncertainty representations that can unlock new agent capabilities such as proactive clarification seeking and shared mental-model building.
arXiv:2610.15046v1 Announce Type: new Abstract: Beliefs in a Partially Observable Markov Decision Process (POMDP) are posterior distributions over the hidden state given the agent's history of action...
arXiv:2607. 02686v1 Announce Type: new Abstract: Reinforcement learning agents operating under partial observability must act on incomplete information, making them natural candidates for guidance from small language models (SLMs) that carry broad reasoning priors.
arXiv:2607. 09748v1 Announce Type: new Abstract: In distributed systems, the classical State Machine Replication (SMR) model assumes that correct replicas execute deterministic transitions to yield identical bitwise states.
arXiv:2605.12718v2 Announce Type: replace Abstract: Multi-agent debate has emerged as a promising approach for improving LLM reasoning on ground-truth tasks, yet current methodologies face certain st...
Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States introduces PoS, an inference-time framework that builds and maintains explicit belief states to guide large language model agents. Each belief state combines an estimate of the current world with unresolved task requirements, making clear what the agent still needs to learn and accomplish. PoS validates consistency, monitors task progress to detect Belief Trapping, and tailors recovery to the trapping pattern and unresolved requirements, achieving top performance across four benchmarks with all three LLM backbones.
VEPA is a framework that enables the verified evolution of formal guardrails for lifelong large language model agents. It represents guardrails as formal policies of symbolic rules over typed agent events, and during deployment uses LLM‑guided feedback to propose policy patches that fix new failures while preserving correct behavior. A deterministic verifier accepts only structurally valid, conflict‑free policies, and an evidence‑aware authority layer activates verified rules only when sufficient supporting evidence exists, ensuring that uncertain updates do not directly affect guardrail decisions.
The paper introduces a Bayesian self‑escalation strategy for hierarchical large‑language‑model agents, allowing an agent to detect during its own reasoning that it is unlikely to succeed and hand control over to a stronger model. The authors formalise this as an optimal‑stopping problem over a learned competence posterior, derive a myopic escalation threshold, and prove that the optimal policy is a time‑varying threshold without assumptions on the raw signal. They provide theoretical guarantees—including a 1/√n regret decay with n calibration trajectories—and validate the approach in simulations and a real‑model code‑generation cascade, showing that the escalation frontier outperforms post‑hoc routing at equal cost. whyItMatters":"The study offers a principled, theoretically grounded method for agents to dynamically decide when to seek stronger models, potentially improving efficiency and reliability in hierarchical LLM systems."
arXiv:2607. 11388v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled increasingly capable digital agents for computer use.
The paper studies how large language models (LLMs) plan under partial observability, comparing aggressive planners that commit to long plans based on unobserved assumptions with conservative planners that only use observed information. Both strategies, whether the LLM outputs a plan directly or generates a PDDL formulation, achieve similar accuracy across three LLMs on two domains. Conservative planners reduce failures but increase API calls, while aggressive planners fail more often but replan efficiently when priors are informative. The study also finds that LLMs treat priors almost as facts, risking failure if priors are unreliable, and proposes hybrid strategies with tunable hyperparameters to balance safety and efficiency.