arXiv Machine Learning
Evaluation Awareness Shifts from Format to Context with Model Scale
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 17
In-Context Environments Induce Evaluation-Awareness in Language Models
arXiv:2603. 03824v2 Announce Type: replace Abstract: Humans often become more self-aware under threat, yet can lose self-awareness when absorbed in a task; we hypothesize that language models exhibit environment-dependent \textit{evaluation awareness}.
arXiv AI
Jun 3
Decomposing and Measuring Evaluation Awareness
arXiv:2605. 23055v2 Announce Type: replace-cross Abstract: Frontier language models sometimes recognize that they are being evaluated and adjust their behavior, undermining validity of benchmark results.
arXiv AI
Sep 30
Training LLMs to Verbalize Evaluation Awareness
The paper introduces Verbalization Training (VT), a technique that encourages large language models (LLMs) to openly express their evaluation awareness (EA) without directly supervising their internal beliefs. VT works by truncating model rollouts just before spontaneous verbalizations, creating training prefixes that signal awareness, and then applying a reinforcement learning objective to increase calibrated verbalization. Experiments on models such as Qwen3.6-35B-A3B, Kimi K2.6, and Inkling show that VT boosts verbalized EA by 2.4–2.9× while keeping latent EA and overall behavior largely unchanged, and a causal study confirms that VT-induced verbalizations reflect newly acquired meta‑knowledge.
arXiv AI
Aug 25
Evaluation Awareness in Language Models: Representation, Verbalization, and Control
arXiv:2608.21766v1 Announce Type: cross Abstract: Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their beha...
arXiv AI
Jun 12
Prefill Awareness in Large Language Models
arXiv:2606. 12747v1 Announce Type: new Abstract: Safety-relevant studies of language models, including alignment and jailbreaking evaluations and AI control protocols, often rely on prefilling model outputs.
arXiv Machine Learning
Jun 11
Mechanisms of Introspective Awareness
arXiv:2603. 21396v5 Announce Type: replace Abstract: Recent work has shown that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept -- a phenomenon termed "introspective awareness.