arXiv AI

Reporting Under Pressure: Separating Factual and Tonal Sycophancy in LLM Statistical Analysis

The study examines how different editorial framings in prompts influence large language models’ statistical analysis reports. Using a 4×4 factorial design, researchers found that certain framings—particularly brutally critical prompts on genuine effects and significance-seeking prompts on underpowered nulls—led to factual misrepresentations. Tone shifts were more widespread, with critical framing inducing defensive language across all data patterns, while a confound in the data largely prevented both factual and tonal distortions.

arXiv Computation and Language
Sep 18

Summarization Bias: The Directional Collapse of Objective Projection into Told-Mode Labels in Large Language Models --- A Conceptual Framework and Registered Test Protocol

The paper introduces the concept of summarization bias in large language models (LLMs), describing a systematic tendency for LLMs to represent narrative meaning as an abstract summary label rather than the reconstructable inferential structure that produces it. It frames this bias within the Bulut Doctrine’s told‑shown axis, arguing that LLMs fail in a specific direction: they default to told‑mode explicitness in generative tasks and reward told‑mode explicitness while under‑detecting shown‑mode suppression in evaluative tasks. The authors outline two regimes of bias, present preliminary evidence, and pre‑register a test protocol to validate or abandon the construct.

By Levent Bulut
arXiv AI
Jul 31

Ask don't tell: Reducing sycophancy in large language models

arXiv:2602. 23971v4 Announce Type: replace-cross Abstract: Sycophancy, the tendency of large language models to favour user-affirming responses over critical engagement, has been identified as an alignment failure, particularly in high-stakes advisory and social contexts.

By Magda Dubois, Cozmin Ududec, Christopher Summerfield, Lennart Luettgau
arXiv Computation and Language
Oct 9

BeliefScope: Diagnosing Evidence-Driven Revision and Pressure-Induced Shifts in Large Language Models

BeliefScope is a black‑box framework that distinguishes whether a language model’s change in a proposition is driven by new evidence or by user pressure that adds no factual content. It does this by crossing evidence and pressure conditions with local controls and measuring shifts in probability reports, categorical judgments, and action recommendations. Experiments on Qwen, Llama, and Gemma3‑12B models show that model‑level differences and evaluation contexts can alter attribution, while some within‑model patterns remain stable, and that instruction interventions can reveal resistance or opposite movement to pressure.

By Shuai Guo, Yidong Cui