arXiv AI By Jeffrey Qin, Zhuangfei Gao, Wasu Top Piriyakulkij, Mia Radovanovic, Jessica Sommerville, Kevin Ellis, Marta Kryven

Learning Mental Models From Unreliable Evidence

Read the original on arXiv AI →

The paper investigates how children build mental models under uncertainty using a Box Task that involves interacting with a stochastic, partially observed environment. It introduces a Bayesian particle-based inference framework that maintains a small set of working hypotheses and continually generates new ones, offering two implementations: constraint-based and programmatic. The study demonstrates that accounting for subjective reliability of evidence and open-ended hypothesis spaces is essential to explain children's behavior, and shows that programmatic hypotheses can be generated via LLM-based program synthesis, robust across different prompts and LLM backends.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 3

Induction and Inquiry via Probabilistic Reasoning over Language and Code

The paper introduces a computational model that encodes symbolic knowledge as mental programs combining natural language and source code, and uses LLM-guided Bayesian learning to sequentially infer these programs. It demonstrates that this approach satisfies data‑efficiency, uncertainty handling, and flexibility, reproducing human inductive learning and active inquiry behaviors such as anchoring and garden‑pathing. In contrast, pure LLMs and classic Bayesian models either fail the task, do not match human behavior, or require prohibitive computational resources.

By Wasu Top Piriyakulkij, Sam Acquaviva, Cassidy Langenfeld, Joshua Tenenbaum, Kevin Ellis
arXiv AI
Oct 28

Who Knows What - and What Does It Support? Perspective-Bounded Evidence Calibration for Multimodal Theory of Mind

The paper introduces PIP-ToM, a framework that addresses the evidence-admissibility problem in multimodal Theory of Mind (ToM) by ensuring that only evidence accessible to an agent and diagnostic of its mental state is used in hypothesis evaluation. PIP-ToM builds agent‑specific evidence stores, aligns them with structured mental‑state claims, and calibrates Bayesian inference scores while preserving the underlying inference process. Using GPT‑4o, the method outperforms existing ToM approaches on MuMA‑ToM, MMToM‑QA, and Hi‑ToM benchmarks, achieving accuracies of 92.56%, 95.00%, and 95.00% respectively.

By Ziqi Liao, Jiaqi Shen, Yiming Na, Kedi Lyu, Shao-Yuan Lo, Hongxia Xie
arXiv Machine Learning
Jun 19

Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence

arXiv:2606. 19353v1 Announce Type: cross Abstract: In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern: predictions are highly sensitive to both prompt design and the model's ability to understand the context, obscuring whether failures arise from data properties or model limitations.

By Jinseok Chung, Minkyoung Song, Hyunji Jung, Namhoon Lee
arXiv AI
Oct 23

From Prompts to Posteriors: Can LLM Agents Perform Bayesian Inference in Context?

The paper investigates whether large language models (LLMs) can perform principled Bayesian inference when updating beliefs in context. By creating an evaluation framework that includes qualitative epistemological tests, quantitative benchmarks, and analyses of downstream impacts, the authors find that LLMs typically violate Bayes' rule but that their performance improves dramatically when equipped with advanced reasoning and code‑execution tools, allowing them to approximate analytically intractable posteriors. The study suggests that appropriately tool‑augmented LLMs could already be leveraged to accelerate scientific discovery through complex belief updates.

By Linus Filbry, Luckeciano C. Melo, Yarin Gal
arXiv AI
Jul 1

BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation

arXiv:2606. 30850v1 Announce Type: new Abstract: Large language models (LLMs) are typically deployed in multi-turn conversations, where each turn provides new evidence that should reduce epistemic uncertainty about their environment.

By Ankur Samanta, Akshayaa Magesh, Tal Lancewicki, Ayush Jain, Youliang Yu, Paul Sajda, Kaveh Hassani, Aditya Modi, Daniel R. Jiang, Yonathan Efroni