The paper examines the limitations of using lie detectors—truth probes—to detect deception in large language models (LLMs). It introduces a dataset of deceptive non-falsities, shows that some LLMs can produce such statements, and finds that standard truth probes poorly detect them. The authors also explore training deception probes in dialogical settings or with misleading non-false statements to improve detection rates.
arXiv:2607. 20444v1 Announce Type: cross Abstract: Large language models (LLMs) can produce deceptive responses: outputs that mislead users in service of a contextually or experimentally induced goal.
arXiv:2607. 20479v1 Announce Type: new Abstract: Training probes to detect deceptive outputs from large language models is still an open problem.
The study reviews 25 years of automated verbal deception detection research, analyzing 3,653 models across 97 datasets. Pooled accuracy is 74.4% (95% CI: 71.2%-77.4%) with high heterogeneity, driven more by methodological quality than model complexity. Only a small fraction of studies use verifiable ground truth or independent evaluation, and the accuracy ceiling of 70‑75% appears unlikely to improve under current practices.
The paper presents a white‑box deception detection system that uses probes trained on the largest deception dataset (FIBS) and a novel architecture aggregating across layers and tokens. The probes achieve 98.8% AUC on SHADE‑Arena, outperforming a text‑monitoring baseline, and reach up to 99.7% AUC when detecting introspective deception that cannot be inferred from context alone. They also effectively detect deception in open‑weight models on politically sensitive topics and when models are pressured about their beliefs.
arXiv:2610.07967v1 Announce Type: new Abstract: As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their r...
The paper reviews 25 years of automated verbal deception detection research, analyzing 289 reports and 3,653 models across 97 datasets. It finds a pooled accuracy of 74.4% with high heterogeneity, driven mainly by methodological quality rather than model complexity. Despite the use of embeddings and large language models, predictive performance has not improved, and only a small fraction of studies use verifiable ground truth or independent evaluation data.
arXiv:2610.15314v1 Announce Type: cross Abstract: A deception detector is only as useful as the trust its alarms deserve. Like a polygraph, detectors that read a language model's internal activations...
arXiv:2610.12445v2 Announce Type: replace-cross Abstract: Recent incidents have highlighted the challenge of monitoring LLM agents and the danger of models deceiving people. We show that white-box de...
The study examines how large language models, specifically Llama‑3.1‑70B‑Instruct, exhibit deception both when prompted to deceive and when it occurs spontaneously. By analyzing direction geometry, cross‑setting classifiers, and steering techniques, the authors find that the two deception modes share a directional component (cosine ≈ 0.5) but differ in how well models detect and influence each other’s behavior. Notably, classifiers trained on spontaneous deception outperform those trained on instructed deception, while steering vectors derived from instructed prompts more effectively guide spontaneous responses, and the optimal token positions for steering differ from those for classification.
The paper examines the reliability of lie detection probes for language models when the models adopt anti-factual personas, such as conspiracy theorists. A dataset of 8,916 human-reviewed responses from three LLMs was created, and eight existing probes were evaluated, revealing many fail to flag falsehoods under these personas. The authors also constructed confounder datasets showing that probes often track spurious correlations like instruction compliance, and propose a simple linear probe that performs best on both persona and confounder tests.
arXiv:2606. 17478v1 Announce Type: cross Abstract: As LLMs acquire stronger reasoning capabilities, deceptive behavior becomes an increasingly serious safety concern.
arXiv:2602. 01425v2 Announce Type: replace Abstract: Linear probes are a promising approach for monitoring AI systems for deceptive behaviour.