arXiv:2608. 15400v1 Announce Type: new Abstract: Large Language Models (LLMs) are notorious for struggling with assessing their own uncertainty, detecting knowledge conflicts, or recognizing when problems exceed their expertise; such limitations inevitably undermine reliability and trust in LLMs.
arXiv:2607. 11881v1 Announce Type: cross Abstract: Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more.
arXiv:2606. 32032v1 Announce Type: cross Abstract: Metacognition is a critical component of intelligence that describes the ability to monitor and regulate one's own cognitive processes.
arXiv:2603. 29693v3 Announce Type: replace Abstract: A robust decision-making process must take into account uncertainty, especially when the choice involves inherent risks.
The paper introduces a new method for measuring metacognitive abilities in large language models (LLMs) without relying on self-reports, instead testing how well models can use knowledge of their internal states. Using two experimental paradigms, the authors find that recent frontier LLMs can assess and use their own confidence when answering factual and reasoning questions, and can anticipate and appropriately employ the answers they would give. The study also shows that these abilities are limited in resolution, context-dependent, differ qualitatively from human metacognition, and vary across models with similar capabilities, suggesting post‑training processes influence metacognitive development.
arXiv:2605. 23055v2 Announce Type: replace-cross Abstract: Frontier language models sometimes recognize that they are being evaluated and adjust their behavior, undermining validity of benchmark results.
The study trains ten open‑weight large language models (LLMs) to predict their own accuracy on factual multiple‑choice questions before answering. Results show that the models’ confidence signals split into two distinct patterns: early in training, confidence aligns with output consistency (how concentrated the answer distribution is), while later, it aligns with true accuracy but only on data similar to the training set. This indicates that calibration training may not universally teach LLMs to detect their own errors.
arXiv:2603.22161v3 Announce Type: replace Abstract: Metacognition -- assessing the quality of one's own cognitive performance -- guides adaptive behavior across species. Substantial research demonstr...
The paper questions whether large language models (LLMs) truly introspect by critiquing recent studies that claim they can detect and report their internal states. It proposes two necessary conditions for genuine introspection: privileged access to internal representations and second‑order computation that distinguishes from first‑order task performance. Re‑examining two existing paradigms, the authors find that apparent introspective abilities can be explained by input‑based classifiers or generic anomaly detection, concluding that current evidence does not support metacognitive monitoring in LLMs.
arXiv:2408. 05568v2 Announce Type: replace Abstract: Large Language Models (LLMs) exhibit potentially harmful biases that reinforce culturally embedded stereotypes, influence moral judgments, or amplify positive evaluations of majority groups.
arXiv:2609.15972v1 Announce Type: cross Abstract: As language models become more capable, long-term collaboration in learning, reasoning, and decision-making calls for a deeper understanding of the p...
The paper introduces Procedural Graphs, a framework that structures procedural knowledge for large language model agents as (procedure, relation, procedure) triplets, analogous to knowledge graphs for factual data. At each decision point, a guidance model uses the local subgraph to bias the agent’s next action, while an LLM refiner self‑evolves the graph by comparing failed and successful trajectories, editing its topology to improve performance. Experiments across various datasets, tasks, and LLMs show that Procedural Graphs consistently outperform memory‑based baselines, and the self‑evolution mechanism further enhances results without manual engineering.
The article presents a minimal working model for large language model (LLM) systems, emphasizing four key distinctions—pretraining vs. deployment, distribution vs. samples, types of memory, and task competence vs. agency. Using this framework, it diagnoses six common misconceptions about LLMs (next‑token prediction, regression to the mean, training‑data regurgitation, model memory, alignment, and understanding), explaining what each misconception captures correctly, where it conflates distinctions, and the implications for evaluation, design, and governance. The model is applied to AI policy language, illustrating how policy can misrepresent these distinctions and offering a diagnostic toolkit to correct such errors.