← Back to all news
arXiv Computation and Language By Yao Huang, Siyuan Chen, Zhenghan Zhou, Yinpeng Dong

Why Static Probes Fail: Context Determines Deception Feature Composition

Read the original on arXiv Computation and Language →

The paper argues that single‑direction probes are insufficient for detecting deception in large language models because deception manifests in multiple, context‑dependent facets. It proposes modeling deception as a multi‑dimensional subspace and uses iterative orthogonal decomposition to uncover complementary directions. A domain‑adaptive gating mechanism then dynamically composes these directions with few‑shot adaptation, achieving high AUROC and recall across diverse datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
Sep 4

Probe Generalization as Subspace Selection for OOD Deception Detection

Linear probes can identify behaviors in language model activations but often fail on out‑of‑distribution data. This study shows that projecting inputs onto a small set of principal components (PCs) from the training distribution allows probes for Llama‑3.1‑8B‑Instruct to transfer across three deception‑detection datasets, nearly matching probes trained directly on the test data. By scoring PCs with an LLM judge to select those that encode transferable deception directions, the authors close the baseline‑to‑oracle gap by 78% on Insider Trading Report and 25% on Sandbagging, revealing that subspace selection largely determines OOD robustness.

By Daniel Yoo, Adrians Skapars
llms
More like this →
arXiv AI
Oct 9

Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception

The paper presents a white‑box deception detection system that uses probes trained on the largest deception dataset (FIBS) and a novel architecture aggregating across layers and tokens. The probes achieve 98.8% AUC on SHADE‑Arena, outperforming a text‑monitoring baseline, and reach up to 99.7% AUC when detecting introspective deception that cannot be inferred from context alone. They also effectively detect deception in open‑weight models on politically sensitive topics and when models are pressured about their beliefs.

By Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba, Adam Gleave, Chris Cundy
llmsagents
More like this →