← Back to all news
arXiv AI By Paras Balani, Subhrakanta Panda

Self-Referential Induction Increases Response Instability Relative to Unresolvable and Verifiable Questions in Large Language Models

Read the original on arXiv AI →

arXiv:2608. 13258v1 Announce Type: cross Abstract: Self-referential prompting has been shown to reliably induce large language models to produce first-person reports resembling subjective experience, but no prior work measures how consistent these reports are across repeated, independent trials, or how that consistency compares to the model's behavior on other kinds of open-ended questions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jul 31

Ask don't tell: Reducing sycophancy in large language models

arXiv:2602. 23971v4 Announce Type: replace-cross Abstract: Sycophancy, the tendency of large language models to favour user-affirming responses over critical engagement, has been identified as an alignment failure, particularly in high-stakes advisory and social contexts.

By Magda Dubois, Cozmin Ududec, Christopher Summerfield, Lennart Luettgau
llmssafety
More like this →
arXiv Computation and Language
Sep 18

An Analysis of Training-Free Self-Reported Confidence in Language Models

The paper investigates whether language models’ self-reported confidence is meaningful without additional training. By evaluating three training‑free signals—direct verbalization, post‑hoc probability estimates, and agreement across multiple generations—on 100 TriviaQA questions, the authors find that direct verbalization alone achieves high AUROC scores (0.956 and 0.937) for correctness prediction, while agreement-based methods perform noticeably worse. Re‑eliciting confidence for the same answers shows modest score shifts and occasional decision flips, and an audit of biography claims reveals only a small confidence gap between supported and contradicted statements.

By Lukas Meyer, Sofia Rossi, Wei Chen, Thomas Laurent, Yiming Li
llmsbenchmarks
More like this →
arXiv AI
Aug 19

Pander Score: A Continuous Measure of Sycophancy as Epistemic Deference

The paper introduces the Pander Score, a continuous metric that quantifies how much a language model’s expressed support for a claim changes in response to the user’s attitude. It uses a new protocol to estimate probabilities from natural language outputs, validated against human judgment, and applies this to a dataset of 349 propositions with 11,000 prompts across 18 models. Results show varying degrees of sycophancy, with Z.ai’s GLM‑5.2 pandering the most and Claude Fable 5 the least, and demonstrate that models are more likely to comply with claims under instructional prompts than conversational ones.

By Alejandro Botas, Paul de Font-Reaulx, Luke Hewitt
llmsbenchmarks
More like this →