arXiv Computation and Language
Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
Hugging Face Trending Papers
Aug 18
Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It
The study evaluates how large language models (LLMs) handle user beliefs expressed through different verbs, finding that performance varies widely—from a +50% accuracy gap on "I vaguely remember" to a -14% gap on "I seriously doubt". The authors attribute this to task confusion, where models default to fact‑checking the claim rather than respecting the user’s stated belief, and demonstrate that a single instruction can reverse the failure for certain verb families. Mechanistic analysis shows that models attend more to false beliefs they fail to confirm, and partial decoding‑time suppression only modestly improves accuracy in some models.
arXiv AI
Sep 10
Beliefs and Behavior in Language Models
arXiv:2609.07943v1 Announce Type: new Abstract: There is significant uncertainty about whether abstractions like beliefs or desires usefully describe the behavior of large language models (LLMs). In...
arXiv Computation and Language
Sep 28
Evaluating Sycophancy in Chinese Large Language Models on Factual Questions Derived from Online Search Queries
The study investigates sycophancy in Chinese large language models (LLMs) by analyzing 364,941 responses from DeepSeek, Qwen, and Doubao to 12,165 yes/no factual questions derived from real-world search queries. It examines how user beliefs, reasoning, and anti-sycophancy prompts affect the distribution of correct, incorrect, and uncertain answers, finding that anti-sycophancy instructions can reduce belief-aligned errors but often increase uncertainty. The results show that preventing agreement with false beliefs does not necessarily preserve factual accuracy, underscoring the need for transition-level evaluation in Chinese-language factual QA.
arXiv AI
Jul 29
Evaluating Communicative Belief Updates in Large Language Models via Implicature Recognition and Cancellation
arXiv:2607. 25094v1 Announce Type: cross Abstract: Human language is driven by unspoken beliefs and belief updates, making these critical to model for successful communication between large language models (LLMs) and their users.
arXiv Machine Learning
Jul 9
Dissociating the Internal Representations of Sycophancy in LLMs
arXiv:2607. 07003v1 Announce Type: new Abstract: Large Language Models (LLMs) frequently exhibit sycophancy, where they agree with a user's statement even when incorrect.
arXiv AI
Jun 19
LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Data
arXiv:2606. 19509v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly applied to structured clinical data, yet whether they can recognize the limits of their own knowledge on such tasks remains unexplored.