Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States introduces PoS, an inference-time framework that builds and maintains explicit belief states to guide large language model agents. Each belief state combines an estimate of the current world with unresolved task requirements, making clear what the agent still needs to learn and accomplish. PoS validates consistency, monitors task progress to detect Belief Trapping, and tailors recovery to the trapping pattern and unresolved requirements, achieving top performance across four benchmarks with all three LLM backbones.
By Yu Luo, Jiamin Jiang, Yimin Zuo, Xidao Wen, Rongchen Gao, Yongqian Sun, Shenglin Zhang, Guiyang Liu, Cheng Zhang, Fang Situ, Qi Zhou, Dan Pei
arXiv:2610.17444v1 Announce Type: cross
Abstract: Language model (LM) agents can fail for different reasons, requiring different explanations and interventions. We study how state beliefs, action val...
By Ayse Sila Okcu, Weisheng Wang, Tongmeng Xie, Niall Dalton, Angelos Nalmpantis, Raghu Arghal, Mario Giulianelli
arXiv:2610.17310v1 Announce Type: new
Abstract: Language-model agents are expected to adapt to others whose behavior changes through interaction. Yet adaptation may rely on learned behavioral pattern...
By Hanbo Xie, Yue Zhao, Robert C. Wilson
The paper introduces LAMB, a recurrent framework that stores latent multi‑hypothesis beliefs for long‑horizon language agents. LAMB separates a shared Common state containing interaction evidence from weighted Delta components representing competing hypotheses, and updates these hypotheses using relative evidence from action‑outcome predictions without generating textual memory. Experiments on ALFWorld, BabyAI, ScienceWorld, and Crafter show that LAMB matches or surpasses prior methods, notably improving success rates and reducing redundant actions.
By Peien Ding, Hui Li, Yuhan Liu, Jinsong Su
arXiv:2512. 20111v2 Announce Type: replace-cross Abstract: As the time horizons of sequential decision-making tasks grow, keeping full interaction histories in model context becomes increasingly costly.
By Aly Lidayan, Jakob Bjorner, Satvik Golechha, Kartik Goyal, Alane Suhr
arXiv:2607. 06503v1 Announce Type: new Abstract: Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable.
By Kai Ruan, Zihe Huang, Ziqi Zhou, Qianshan Wei, Xuan Wang, Hao Sun
Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable. We show that failure is predictable early from the agent's internal representations: lightweight per-round probes on hidden activations anticipate eventual episode failure as early as the first interaction round, where scorers reading only the agent's observable behavior are barely better than chance.
The paper introduces the Belief-State Engine (BSE), an inference module that supplies a large language model (LLM) with a Bayesian posterior over hidden states in a partially observable Markov decision process (POMDP). By keeping the raw action‑observation log hidden from the LLM, the BSE ensures the agent behaves as a sound Markov policy on the belief MDP, thereby inheriting classical POMDP optimality guarantees. Experiments on the Tiger POMDP and a red‑team attack‑graph task show that BSE‑augmented agents outperform six baselines in task return, belief calibration, and decision consistency.
By Arnab Chattopadhayay, Debdipta Halder
The study evaluates how large language models (LLMs) handle user beliefs expressed through different verbs, finding that performance varies widely—from a +50% accuracy gap on "I vaguely remember" to a -14% gap on "I seriously doubt". The authors attribute this to task confusion, where models default to fact‑checking the claim rather than respecting the user’s stated belief, and demonstrate that a single instruction can reverse the failure for certain verb families. Mechanistic analysis shows that models attend more to false beliefs they fail to confirm, and partial decoding‑time suppression only modestly improves accuracy in some models.
arXiv:2609.00455v1 Announce Type: new
Abstract: Large language models (LLMs) are being used as policies for autonomous decision-making and planning in many domains. Despite their strong reasoning cap...
By Shubham Kumar, Harshit Kumar, Narendra Ahuja, Saurabh Jha
Process Reward Models (PRMs) are crucial for scaling large language model reasoning at test time, but they suffer from two types of uncertainty: PRM uncertainty from limited supervision and imperfect training, and policy uncertainty from mismatched data collection policies. The paper introduces BayesianPRM, which treats process rewards as random variables, discretizes the reward space with an ensemble, and applies two belief updates—reliability‑aware updating for PRM uncertainty and conservatism‑aware calibration for policy uncertainty—to form a Bayesian belief whose expectation yields a more accurate reward estimate. Experiments on multimodal reasoning benchmarks show that BayesianPRM improves reward accuracy and test‑time scaling performance.
By Zhongjian Qiao, Zhixiang Zhou, Jiafei Lyu, Peisong Wang, Qiaosheng Zhang, Siyang Gao, Shuang Qiu
arXiv:2607. 02686v1 Announce Type: new Abstract: Reinforcement learning agents operating under partial observability must act on incomplete information, making them natural candidates for guidance from small language models (SLMs) that carry broad reasoning priors.
By Juarez Monteiro, Nathan Gavenski, Guilherme Lima, Francisco Galuppo, Odinaldo Rodrigues, Adriano Veloso