← Back to all news
arXiv Machine Learning By Elad David, Max Fomin, Amit LeVi

Latent Undertow: How Ordinary Typos Break Probes

Read the original on arXiv Machine Learning →

The paper demonstrates that ordinary typos can significantly disrupt probes that detect malicious prompts in large language models by rotating and decaying the readout vector. A single-position probe’s true positive rate drops by 12 percentage points when three common typos are stacked, and even advanced aggregators only partially mitigate the effect. Introducing a KV‑cache fork—appending a short suffix after the user message—recovers 95% of the lost detection capability, outperforming perturbation‑augmented training by an order of magnitude.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 5

Prompted to Discriminate: Generalizing Malicious-Input Probes in the Wild

The paper investigates whether adding a short classification instruction after a user’s prompt improves the ability of activation probes to detect malicious inputs in large language models. Across 13 safety benchmarks and three open‑weight model families, a classification suffix consistently boosts out‑of‑distribution detection (up to ~4 AUC points) compared to no suffix, and the benefit transfers to multi‑position pooling probes used in production. The improvement stems from the classification format itself rather than the specific content of the instruction, though the optimal suffix varies with the model and readout type.

By Elad David, Max Fomin
llmsagentsbenchmarkssafety
More like this →
arXiv AI
Oct 16

FlashAttention with Statistics: Low-Overhead Attention Features for Compact Hallucination Detectors

The paper introduces FlashAttention with Statistics, a method that computes attention‑based detector features without materializing the full attention matrix, thereby reducing memory usage and latency. By accumulating statistics during the FlashAttention pass or via sparse lookups, the approach adds only a small overhead—16% at 2K tokens and 1.4% at 35K tokens on an H100 GPU—and stores just 22.5 KiB per response token. Experiments show that frozen Qwen3 proxy models, even as small as 0.6 B parameters, can serve as strong hallucination detectors, outperforming larger zero‑shot judges and specialized grounding checkers on several benchmarks, with attention features further boosting performance.

By Andrei Volodichev, Maksim Savichev, Nikita Sviridov, Artem Yuliy, Dmitry Simakov, Alexey Zaytsev
llmsbenchmarkssafety
More like this →