Hugging Face Trending Papers

A persistent accuracy ceiling in automated verbal deception detection

The paper reviews 25 years of automated verbal deception detection research, analyzing 289 reports and 3,653 models across 97 datasets. It finds a pooled accuracy of 74.4% with high heterogeneity, driven mainly by methodological quality rather than model complexity. Despite the use of embeddings and large language models, predictive performance has not improved, and only a small fraction of studies use verifiable ground truth or independent evaluation data.

arXiv Computation and Language
Oct 9

A persistent accuracy ceiling in automated verbal deception detection

The study reviews 25 years of automated verbal deception detection research, analyzing 3,653 models across 97 datasets. Pooled accuracy is 74.4% (95% CI: 71.2%-77.4%) with high heterogeneity, driven more by methodological quality than model complexity. Only a small fraction of studies use verifiable ground truth or independent evaluation, and the accuracy ceiling of 70‑75% appears unlikely to improve under current practices.

By Riccardo Loconte, Jonas Festor, Zane Fatjanova, Mariam Bolkvadze, Bennett Kleinberg
arXiv AI
Sep 2

Asymmetries in Spontaneous and Instructed Deception

The study examines how large language models, specifically Llama‑3.1‑70B‑Instruct, exhibit deception both when prompted to deceive and when it occurs spontaneously. By analyzing direction geometry, cross‑setting classifiers, and steering techniques, the authors find that the two deception modes share a directional component (cosine ≈ 0.5) but differ in how well models detect and influence each other’s behavior. Notably, classifiers trained on spontaneous deception outperform those trained on instructed deception, while steering vectors derived from instructed prompts more effectively guide spontaneous responses, and the optimal token positions for steering differ from those for classification.

By Josiah Luikham
arXiv AI
Oct 9

Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception

The paper presents a white‑box deception detection system that uses probes trained on the largest deception dataset (FIBS) and a novel architecture aggregating across layers and tokens. The probes achieve 98.8% AUC on SHADE‑Arena, outperforming a text‑monitoring baseline, and reach up to 99.7% AUC when detecting introspective deception that cannot be inferred from context alone. They also effectively detect deception in open‑weight models on politically sensitive topics and when models are pressured about their beliefs.

By Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba, Adam Gleave, Chris Cundy