arXiv Computation and Language
The study investigates whether prompting language models to provide five weighted estimates in a single response can reduce bias in their judgments. Across various bias tasks—anchoring, hindsight, base‑rate, conjunction, and status‑quo/default—the authors compare direct numerical answers with a multi‑estimate format, testing different source framings, random cues, and probability‑consistency constraints on three models (gpt‑6.1‑sol, claude‑sonnet‑5.5, and Qwen3‑8B). Results show that while direct elicitation reduces irrelevant‑anchor indices and status‑quo effects in two configurations, multi‑estimate elicitation does not consistently debias or attenuate responses, and the observed effects are limited by noise and lack of equivalence.
The study investigates whether large language models (LLMs) that allocate extra computation during inference—termed reasoning models—reduce classic decision biases compared to their non‑reasoning counterparts. Using 30 vignettes covering six cognitive biases and varying token budgets up to 8,192 tokens, the authors find that reasoning models are not less biased, and increased deliberation does not reliably diminish bias magnitude. Only anchoring showed a bias in the human direction, while other biases either remained unchanged or moved further from human patterns, suggesting that test‑time reasoning does not guarantee rationality.
By Obada Kraishan
The paper investigates whether language models that produce the same probabilistic predictions after different types of evidence (textual or activation interventions) actually share the same underlying belief states. Using a Bernoulli inference task and an analytic Bayesian reference, the authors compare four textual routes and two activation-level interventions across three open-weight models, measuring both next-outcome continuations and explicit probabilistic estimates. They find that the format of evidence influences which readouts reflect its content, and that the effective evidence weight—how many new observations a belief is worth—varies widely across routes, with some elicitation questions altering inferred weights by orders of magnitude. In a one-shot steering scenario, an intervention that matches a natural record’s immediate prediction loses most of its effect when new evidence is introduced, demonstrating that agreement in current predictions does not guarantee equivalent belief updating.
By Weihan Li, Junhao Wu, Xinlei Chen, Tianshi Zheng
arXiv:2610.27745v1 Announce Type: cross
Abstract: Large language models (LLMs) are increasingly used as judgment assistants and as stand-in participants in psychology experiments, yet little is known...
By An-Sheng Jhang
Large language models (LLMs) are increasingly used as judgment assistants and as stand-in participants in psychology experiments, yet little is known about how much their probability estimates move in...
Language models can produce similar probabilistic predictions after observing outcomes, reading summaries, receiving instructions, or undergoing activation interventions. Whether these routes induce e...
arXiv:2608. 14320v1 Announce Type: new Abstract: The anchoring effect is a cognitive bias in which an initial reference value shifts a later judgment toward itself.
By Yiderigun Borjigin, Alexander Hermann, Christian Cyron, Roland Aydin
Large language models (LLMs) are increasingly used for clinical reasoning, yet their ability to revise judgments as patient evidence changes is uncertain. In this study, researchers evaluated longitudinal belief updating using real intensive‑care patient trajectories and found that conditioning on prior judgments often increased prediction error. Controlled experiments revealed two failure modes: models were more responsive to worsening than improving respiratory evidence, and they shifted estimates significantly when prior risk was altered, indicating a causal influence of prior beliefs. Prompting did not improve reliability, and an Evidence‑Validated Longitudinal Update (EVLU) approach produced fewer but more trustworthy revisions, highlighting a reliability‑coverage trade‑off.
By Min Zeng, Rui Zhang
arXiv:2608.28018v1 Announce Type: cross
Abstract: Knowledge-intensive reasoning requires Large Language Models (LLMs) to ground answers in provided evidence. When evidence is insufficient, it is desi...
By Vy Nguyen, Ziqi Xu, Jeffrey Chan, Estrid He, Feng Xia, Renqiang Luo, Erik Cambria, Xiuzhen Zhang
arXiv:2606. 24281v1 Announce Type: cross Abstract: Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success.
By Conor Finlay, Joshua Kurien, Saurabh Dash, Marzieh Fadaee, Beyza Ermis
arXiv:2605. 27752v2 Announce Type: replace Abstract: LLM confidence calibration is often evaluated by comparing two signals: token-probability scores and verbalized confidence.
By Hankyeol Kim, Pilsung Kang
The study examines how different editorial framings in prompts influence large language models’ statistical analysis reports. Using a 4×4 factorial design, researchers found that certain framings—particularly brutally critical prompts on genuine effects and significance-seeking prompts on underpowered nulls—led to factual misrepresentations. Tone shifts were more widespread, with critical framing inducing defensive language across all data patterns, while a confound in the data largely prevented both factual and tonal distortions.
By Paras Balani, Subhrakanta Panda
arXiv:2608. 11138v1 Announce Type: cross Abstract: We propose that a model's uncertainty about a token is reflected not only in the breadth of its output distribution but also in whether a confident prediction is \emph{fragile} under perturbation of its attention pathways.
By Minsoo Kim, Sungyoung Ji, Kisung Moon, Ilyong Yoon