The paper investigates whether large language models (LLMs) can control their own internal representations, a question relevant to machine metacognition and AI safety. Previous work claimed such control using neurofeedback, but the targets were not privileged, allowing third‑party inference from prompts. By redesigning the paradigm to enforce privileged access—mirroring human neurofeedback experiments—the authors find that LLMs fail to reliably control these internal representations, indicating that earlier claims may have relied on superficial mechanisms.
Whether large language models (LLMs) can control their own internal representations matters for both machine metacognition and AI safety. A recent study applied neurofeedback to LLMs and claimed that...
The paper questions whether large language models (LLMs) truly introspect by critiquing recent studies that claim they can detect and report their internal states. It proposes two necessary conditions for genuine introspection: privileged access to internal representations and second‑order computation that distinguishes from first‑order task performance. Re‑examining two existing paradigms, the authors find that apparent introspective abilities can be explained by input‑based classifiers or generic anomaly detection, concluding that current evidence does not support metacognitive monitoring in LLMs.
arXiv:2610.20179v1 Announce Type: cross Abstract: Probes on activation vectors are widely used to detect misbehavior in language models, but their success relies on the assumption that models cannot...
Probes on activation vectors are widely used to detect misbehavior in language models, but their success relies on the assumption that models cannot control what their activations encode. Humans can l...
The paper introduces a new method for measuring metacognitive abilities in large language models (LLMs) without relying on self-reports, instead testing how well models can use knowledge of their internal states. Using two experimental paradigms, the authors find that recent frontier LLMs can assess and use their own confidence when answering factual and reasoning questions, and can anticipate and appropriately employ the answers they would give. The study also shows that these abilities are limited in resolution, context-dependent, differ qualitatively from human metacognition, and vary across models with similar capabilities, suggesting post‑training processes influence metacognitive development.
arXiv:2606. 05976v2 Announce Type: replace Abstract: Recent works show that LLM agents struggle to correct errors in their own reasoning traces, despite their ability to correct errors from external sources.
arXiv:2607. 25907v1 Announce Type: cross Abstract: Activation steering controls model behavior by editing internal activations at inference time.
arXiv:2408. 05568v2 Announce Type: replace Abstract: Large Language Models (LLMs) exhibit potentially harmful biases that reinforce culturally embedded stereotypes, influence moral judgments, or amplify positive evaluations of majority groups.
arXiv:2606. 05976v1 Announce Type: new Abstract: Recent work shows that LLM agents struggle to correct errors in their own reasoning traces yet show markedly higher correction rates when identical claims appear under external sources.
The paper argues that effective oversight of large language models (LLMs) depends on users’ ability to retrieve relevant information during review. Through two lab-in-the-field experiments with 640 customer‑facing employees, the authors demonstrate that self‑generated explanations and retrieval cues improve error detection and sustain it over repeated LLM use. They propose that information retrievability is a distinct precondition for oversight, and suggest lightweight onboarding explanations and daily cues as practical solutions.
arXiv:2603.22161v3 Announce Type: replace Abstract: Metacognition -- assessing the quality of one's own cognitive performance -- guides adaptive behavior across species. Substantial research demonstr...
The paper investigates how language models (LMs) can introspect their internal signals and identifies a problem called causal bypass, where self-reports depend on correlated but causally unrelated signals. By training hundreds of introspectors on low‑level signals across multiple LMs, the authors show that standard supervised fine‑tuning (SFT) often leads to causal bypass. They propose interventional training, which trains models to report on counterfactually perturbed values, and demonstrate that this approach yields more accurate introspection for both endogenous and injected signals, even on out‑of‑distribution data, and that the method generalizes across token positions and scales with more data.