arXiv AI

BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation

arXiv:2606. 30850v1 Announce Type: new Abstract: Large language models (LLMs) are typically deployed in multi-turn conversations, where each turn provides new evidence that should reduce epistemic uncertainty about their environment.

arXiv AI
Sep 2

HugAgent: A Human Simulation Benchmark for Individual-Level Reasoning

arXiv:2510.15144v4 Announce Type: replace Abstract: Simulating human reasoning in open-ended tasks has long been a central aspiration in AI and cognitive science. While large language models now appr...

By Chance Jiajie Li, Zhenze Mo, Yuhan Tang, Ao Qu, Jiayi Wu, Kaiya Ivy Zhao, Yulu Gan, Jie Fan, Jiangbo Yu, Hang Jiang, Paul Pu Liang, Jinhua Zhao, Luis Alberto Alonso Pastor, Kent Larson
arXiv AI
Sep 21

Bayesian Belief Layer for Controllable Opinion Dynamics in LLM Agents

The paper introduces Bayesian Chronicle Agents (BCA), a lightweight belief layer that separates an LLM agent’s internal stance from its outward speech. Each stance is represented as a probability updated via a single Bayesian step per utterance, with a single prior‑strength parameter κ controlling stubbornness. By sweeping κ, the authors generate three controllable opinion‑dynamics regimes—consensus, persistent disagreement, and committed‑minority influence—matching Friedkin–Johnsen theory and demonstrating recoverable, auditable belief states across models.

By Hafsa Akbar, Daniel Platnick, Marjan Alirezaie, Hossein Rahnama
arXiv AI
Jul 10

What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness

arXiv:2607. 08046v1 Announce Type: cross Abstract: Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast.

By Rapha\"el Sarfati, Pratyush Ranjan Tiwari, Siddharth Boppana, Christopher J. Earls, Srikar Varadaraj, Eric Ho
arXiv AI
Sep 7

Evidence Integration in Large Language Models

The paper proposes a distributional theory explaining how large language models (LLMs) incorporate external evidence into their decision-making process. It identifies three key predictions: (1) evidence is more persuasive when it aligns with the model’s prior beliefs, (2) models more readily accept errors from their own internal processes than from external sources, and (3) the same evidence can improve weaker models while harming stronger ones. Extensive experiments across ten million trials, twelve LLMs from four families, and eight domains—including quantum mechanics, physics, genetics, and molecular biology—confirm these predictions and reveal that evidence integration occurs late in the network as a structured sequence of steps rather than through a simple trust metric.

By Sebastien Kawada, Manolis Kellis
arXiv AI
Sep 10

Beliefs and Behavior in Language Models

arXiv:2609.07943v1 Announce Type: new Abstract: There is significant uncertainty about whether abstractions like beliefs or desires usefully describe the behavior of large language models (LLMs). In...

By Alex Smolin, Bryan Wilder
arXiv AI
Sep 3

Induction and Inquiry via Probabilistic Reasoning over Language and Code

The paper introduces a computational model that encodes symbolic knowledge as mental programs combining natural language and source code, and uses LLM-guided Bayesian learning to sequentially infer these programs. It demonstrates that this approach satisfies data‑efficiency, uncertainty handling, and flexibility, reproducing human inductive learning and active inquiry behaviors such as anchoring and garden‑pathing. In contrast, pure LLMs and classic Bayesian models either fail the task, do not match human behavior, or require prohibitive computational resources.

By Wasu Top Piriyakulkij, Sam Acquaviva, Cassidy Langenfeld, Joshua Tenenbaum, Kevin Ellis