The paper questions whether large language models (LLMs) truly introspect by critiquing recent studies that claim they can detect and report their internal states. It proposes two necessary conditions for genuine introspection: privileged access to internal representations and second‑order computation that distinguishes from first‑order task performance. Re‑examining two existing paradigms, the authors find that apparent introspective abilities can be explained by input‑based classifiers or generic anomaly detection, concluding that current evidence does not support metacognitive monitoring in LLMs.
By Shashwat Singh, Tal Linzen, Shauli Ravfogel
arXiv:2603.22161v3 Announce Type: replace
Abstract: Metacognition -- assessing the quality of one's own cognitive performance -- guides adaptive behavior across species. Substantial research demonstr...
By Dharshan Kumaran, Nathaniel Daw, Simon Osindero, Petar Veli\v{c}kovi\'c, Viorica Patraucean
arXiv:2607. 11881v1 Announce Type: cross Abstract: Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more.
By Gabrielle Kaili-May Liu, Areeb Gani, Jacqueline Lu, Jordan Thomas, Mark Steyvers, Arman Cohan
arXiv:2603.17839v4 Announce Type: replace-cross
Abstract: Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from...
By Dharshan Kumaran, Arthur Conmy, Federico Barbero, Simon Osindero, Viorica Patraucean, Petar Veli\v{c}kovi\'c
The study trains ten open‑weight large language models (LLMs) to predict their own accuracy on factual multiple‑choice questions before answering. Results show that the models’ confidence signals split into two distinct patterns: early in training, confidence aligns with output consistency (how concentrated the answer distribution is), while later, it aligns with true accuracy but only on data similar to the training set. This indicates that calibration training may not universally teach LLMs to detect their own errors.
By Nicolas Yax, Stefano Palminteri, Pierre-Yves Oudeyer
arXiv:2606. 32032v1 Announce Type: cross Abstract: Metacognition is a critical component of intelligence that describes the ability to monitor and regulate one's own cognitive processes.
By Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona, Idan Szpektor, Arman Cohan
Whether large language models (LLMs) can control their own internal representations matters for both machine metacognition and AI safety. A recent study applied neurofeedback to LLMs and claimed that...
The paper investigates whether large language models (LLMs) can control their own internal representations, a question relevant to machine metacognition and AI safety. Previous work claimed such control using neurofeedback, but the targets were not privileged, allowing third‑party inference from prompts. By redesigning the paradigm to enforce privileged access—mirroring human neurofeedback experiments—the authors find that LLMs fail to reliably control these internal representations, indicating that earlier claims may have relied on superficial mechanisms.
By Koshiro Aoki, Ryota Takatsuki, Gouki Minegishi, Yusuke Haruki, Daisuke Kawahara
arXiv:2610.07186v1 Announce Type: new
Abstract: Large language models make claims about themselves that are both consequential and increasingly difficult to verify from behavior alone. How can we dis...
By David I. Atkinson, Dillon Plunkett, David Bau
arXiv:2608. 15400v1 Announce Type: new Abstract: Large Language Models (LLMs) are notorious for struggling with assessing their own uncertainty, detecting knowledge conflicts, or recognizing when problems exceed their expertise; such limitations inevitably undermine reliability and trust in LLMs.
By Charles Courchaine, Ricky J. Sethi, Hefei Qiu
arXiv:2603. 25112v2 Announce Type: replace-cross Abstract: Standard evaluation of LLM confidence relies on calibration metrics (ECE, Brier score) that conflate how much a model knows (Type-1 accuracy) with how well its confidence signal tracks that knowledge (Type-2 metacognitive sensitivity).
By Jon-Paul Cacioli
The paper presents the first dataset that captures how humans learn and use mechanisms in unfamiliar environments, using ARC‑AGI‑3 as a testbed. It includes 144 trajectories from 64 participants across 25 public games, each paired with time‑synchronized actions and think‑aloud audio, and 250 trajectories from large language model agents with and without harnesses. The dataset is annotated for mechanism triggers, action purposes, errors, and recovery, enabling a comparison of human and LLM cognition, highlighting differences in hypothesis testing, exploration, and use of qualitative representations.
By Shuzhi Liu, Archana Warrier, Dat Nguyen, Chengqi Zheng, Kevin Ellis, Yewen Pu