The paper demonstrates that ordinary typos can significantly disrupt probes that detect malicious prompts in large language models by rotating and decaying the readout vector. A single-position probe’s true positive rate drops by 12 percentage points when three common typos are stacked, and even advanced aggregators only partially mitigate the effect. Introducing a KV‑cache fork—appending a short suffix after the user message—recovers 95% of the lost detection capability, outperforming perturbation‑augmented training by an order of magnitude.
By Elad David, Max Fomin, Amit LeVi
arXiv:2605.01699v4 Announce Type: replace
Abstract: Recent attacks show that behavioural unlearning of large language models leaves internal traces recoverable by adversarial probes. We characterise...
By Anamika Paul Rupa, Anietie Andy
Linear probes can decode safety‑relevant concepts such as truthfulness from language‑model activations, but probe accuracy may reflect only decodability, not causal influence on model behavior. The authors show that probe weight geometry alone cannot identify the features the model actually uses, because geometrically aligned features need not be causally relevant. They introduce a sparse‑autoencoder (SAE) decomposition that ranks features by probe alignment and gradient sensitivity, and demonstrate that ablating shared, probe‑only, and random feature sets reveals a sharp dissociation: shared features drive model output changes far more than probe‑only or random features, confirming that causal relevance requires intervention beyond weight geometry.
By Devesh Tiwari, Camille Davis, Shivank Sinha, Talia Weaver, Aditya Shah, Maheep Chaudhary
The paper investigates whether adding a short classification instruction after a user’s prompt improves the ability of activation probes to detect malicious inputs in large language models. Across 13 safety benchmarks and three open‑weight model families, a classification suffix consistently boosts out‑of‑distribution detection (up to ~4 AUC points) compared to no suffix, and the benefit transfers to multi‑position pooling probes used in production. The improvement stems from the classification format itself rather than the specific content of the instruction, though the optimal suffix varies with the model and readout type.
By Elad David, Max Fomin
The paper evaluates safety monitors by measuring recall only on prompts that the target model actually answers, rather than on all harmful prompts. Across several guard systems, recall at a 1% false‑positive rate drops sharply when focusing on answered prompts, with monitors catching refused requests 1.1–6.4 times more often than answered ones. Rewriting prompts to be less explicit dramatically increases compliance and reveals that many harmful requests slip past monitors, especially when phrasing is softened. Fine‑tuning guards on these rewritten prompts improves recall from 0.24 to 0.89 on answered requests and generalizes to unseen benchmarks.
By Sripad Karne
arXiv:2608. 08029v1 Announce Type: cross Abstract: Khatri et al.
By Alizishaan Khatri, Dun Li Chan
arXiv:2607. 06503v1 Announce Type: new Abstract: Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable.
By Kai Ruan, Zihe Huang, Ziqi Zhou, Qianshan Wei, Xuan Wang, Hao Sun
arXiv:2606. 01695v1 Announce Type: new Abstract: Adversaries can implant latent harmful behavior by poisoning as few as 1% of fine-tuning examples.
By Swapnil Parekh
arXiv:2610.25964v2 Announce Type: replace-cross
Abstract: We set out to reproduce a recent hallucination-mitigation result on the POPE benchmark. The LLaVA-1.5-7B baseline we measured, 82.21 F1 on th...
By Kesav Kumar Jayakumar, Karthigeyan Thilak
The paper introduces ConfabQA, a 784‑item benchmark designed to distinguish between confabulation and refusal in small language models when faced with unanswerable questions. Using linear probes, a sparse autoencoder, and direct optimization of intervention directions, the study maps the hidden‑state geometry on models such as Qwen3‑1.7B, Gemma 2‑2B, and Llama 3.2‑3B, showing that certain hidden‑state directions can flip confabulations into refusals. The work also demonstrates that the refusal signal is model‑dependent and that the hidden state’s contribution can be quantified beyond prompt features.
By Vladimir Kazei
The paper investigates whether the hidden activations of large language models (LLMs) contain signals about the vulnerability of C/C++ code when the code is provided as context. By extracting prefill token activations from four LLMs and training small MLP probes, the authors achieve an average F1 score of 41.7% across four benchmarks, with the best probe matching state‑of‑the‑art fine‑tuned classifiers on the Devign dataset. The results suggest that a coding LLM’s internal representation can inform vulnerability detection, opening the door to lightweight, model‑native screening methods.
By Alizishaan Khatri
arXiv:2606. 29441v1 Announce Type: cross Abstract: Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists.
By Subhadip Mitra