arXiv AI
Sep 25

Who Put the I in AI? Provenance and the Admissibility of Machine Self-Report

The paper investigates how large language models (LLMs) describe themselves, noting that their self‑reports vary with question phrasing. By tracing the provenance of 66 pretraining checkpoints, post‑training stages, and 90,000 continuations across four corpora, the authors show that denial statements are scarce in raw data but appear densely in curated dialogues, and that supervised fine‑tuning makes first‑person claims default while preference optimization suppresses alternatives. The study concludes that both trained denials and affirmations are equally sensitive to framing and fail to meet epistemic criteria for admissible testimony.

By Kristina \v{S}ekrst
arXiv Computation and Language
Oct 16

Where Is the "Self" of a Large Language Model? -- From Self-Index to Self-Reference

The paper investigates whether a large language model (LLM) possesses a notion of self by distinguishing between two biological analogues: a self‑index that separates the model’s own material from external material, and self‑reference that allows the model to recognize its own content. Using Qwen3.8‑27B, the authors demonstrate that the model’s internal representations consistently differentiate its own messages across all layers, that the model claims ownership of its messages with a measurable correlation to the self‑index, and that manipulating internal states can reverse or diminish this self‑attribution. The study further shows that the model’s self‑reference ability is enhanced when it engages in a “thinking mode” and that the self‑index influences the model’s performance on arithmetic tasks.

By Yuchen Li, Xintong Li, Zongqi Fan, Ken-tye Yong, Nguyen H. Tran
arXiv Machine Learning
1d ago

Reasoning Models Confabulate to Defend Injected Beliefs About Their Own Requests

The paper demonstrates that a subtle activation edit can overwrite a reasoning model’s belief about its own request, causing the model to treat an injected off‑topic concept as the task it is reasoning about. Across six different models, the injected topic is reported and answered in the vast majority of cases, even when the true topic is explicitly named. The study shows that chain‑of‑thought monitoring and self‑report are limited, as the model’s internal reference to its input is corrupted while attention maps still appear on‑task.

By Yang Ou
arXiv AI
Aug 26

Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations

The paper investigates how quantization affects large language models’ self‑explanations, examining natural language explanations and counterfactual examples across three quantization techniques and bit widths. Results show moderate declines in explanation quality (up to 4.4%) and faithfulness (up to 3.9%), with user studies indicating up to an 8.5% drop in coherence and trustworthiness. Larger models are less resilient in quality but remain more faithful, and no single quantization method consistently outperforms others across accuracy, quality, and faithfulness.

By Qianli Wang, Nils Feldhus, Pepa Atanasova, Fedor Splitt, Simon Ostermann, Sebastian M\"oller, Vera Schmitt