arXiv Machine Learning

Do LLMs have core beliefs?

The paper investigates whether Large Language Models (LLMs) possess core beliefs—fundamental truths that underpin a stable worldview. Using Adversarial Dialogue Trees across science, history, geography, biology, and mathematics, the authors find that most LLMs cannot maintain consistent commitments, even though newer models show some improvement. The study concludes that current LLMs lack a key component of human-level cognition.

arXiv AI
Sep 10

Beliefs and Behavior in Language Models

arXiv:2609.07943v1 Announce Type: new Abstract: There is significant uncertainty about whether abstractions like beliefs or desires usefully describe the behavior of large language models (LLMs). In...

By Alex Smolin, Bryan Wilder
arXiv AI
Aug 25

DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories

DialToM is a Theory of Mind benchmark created from naturalistic human-human dialogues, using a multiple-choice format. It introduces a State-Driven Diagnostic Probe that requires models to predict dialogue trajectories based solely on isolated mental-state profiles, without dialogue context. The evaluation shows that large language models are good at inferring mental states (Literal ToM) but struggle to use them for social forecasting (Functional ToM), while a domain expert scores 100% accuracy, highlighting a clear human‑AI gap.

By Neemesh Yadav, Palakorn Achananuparp, Jing Jiang, Ee-Peng Lim
arXiv Computation and Language
3d ago

Latent Debate: A Surrogate Framework for Interpreting LLM Thinking towards Binary Decisions

The paper introduces latent debate, a structured surrogate framework that interprets the internal thinking of Large Language Models (LLMs) on True/False tasks by modeling latent arguments and their interactions. It provides a model- and task-agnostic conceptual framework, instantiated symbolically to approximate LLM decision-making. Empirical results show that latent debate closely matches LLM predictions, offers interpretability, and serves as a strong baseline for hallucination detection, revealing correlations between debate patterns and hallucination risk.

By Lihu Chen, Xiang Yin, Francesca Toni
Hugging Face Trending Papers
Aug 12

Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs

Persuasion is a core dynamic of natural language communication, shaping how large language models (LLMs) update beliefs, resolve disagreements, and reach decisions. As LLMs increasingly debate, advise, and think collaboratively with humans and each other, resistance to harmful persuasion becomes a core requirement for reliable behavior.

arXiv Machine Learning
Oct 8

Talking with Language Models

The paper argues that large language models (LLMs) should not be viewed as conversational partners but as sophisticated text-generating artifacts. It introduces the concept of an artifactual stance, framing human‑AI interaction as exchanges of candidate texts optimized for utility rather than meaningful utterances. By rejecting the illusion of dialogue, the authors suggest that philosophical concerns about identity, responsibility, and truth in AI interactions are misplaced, and instead focus should shift to the normative design and use of these artifacts.

By James Ravi Kirkpatrick, Alexandru Radulescu, Rachel Katharine Sterken
arXiv AI
Aug 19

Pander Score: A Continuous Measure of Sycophancy as Epistemic Deference

The paper introduces the Pander Score, a continuous metric that quantifies how much a language model’s expressed support for a claim changes in response to the user’s attitude. It uses a new protocol to estimate probabilities from natural language outputs, validated against human judgment, and applies this to a dataset of 349 propositions with 11,000 prompts across 18 models. Results show varying degrees of sycophancy, with Z.ai’s GLM‑5.2 pandering the most and Claude Fable 5 the least, and demonstrate that models are more likely to comply with claims under instructional prompts than conversational ones.

By Alejandro Botas, Paul de Font-Reaulx, Luke Hewitt