The paper demonstrates that ordinary typos can significantly disrupt probes that detect malicious prompts in large language models by rotating and decaying the readout vector. A single-position probe’s true positive rate drops by 12 percentage points when three common typos are stacked, and even advanced aggregators only partially mitigate the effect. Introducing a KV‑cache fork—appending a short suffix after the user message—recovers 95% of the lost detection capability, outperforming perturbation‑augmented training by an order of magnitude.
arXiv:2610.15883v1 Announce Type: cross Abstract: Text wrapped in a trigger can be made behaviorally invisible to a language model: omitted from summaries and quotations, ignored in question answerin...
The paper investigates whether adding a short classification instruction after a user’s prompt improves the ability of activation probes to detect malicious inputs in large language models. Across 13 safety benchmarks and three open‑weight model families, a classification suffix consistently boosts out‑of‑distribution detection (up to ~4 AUC points) compared to no suffix, and the benefit transfers to multi‑position pooling probes used in production. The improvement stems from the classification format itself rather than the specific content of the instruction, though the optimal suffix varies with the model and readout type.
arXiv:2606. 31168v1 Announce Type: cross Abstract: We audit a fixed prefix-window mean-NLL memorization probe (K=20) on a Qwen2.
The paper introduces FlashAttention with Statistics, a method that computes attention‑based detector features without materializing the full attention matrix, thereby reducing memory usage and latency. By accumulating statistics during the FlashAttention pass or via sparse lookups, the approach adds only a small overhead—16% at 2K tokens and 1.4% at 35K tokens on an H100 GPU—and stores just 22.5 KiB per response token. Experiments show that frozen Qwen3 proxy models, even as small as 0.6 B parameters, can serve as strong hallucination detectors, outperforming larger zero‑shot judges and specialized grounding checkers on several benchmarks, with attention features further boosting performance.
arXiv:2608. 08029v1 Announce Type: cross Abstract: Khatri et al.
arXiv:2606. 30449v1 Announce Type: new Abstract: Probes on model internals could help monitor agentic systems if they identify harmful text or tool actions before those actions are generated.
arXiv:2602. 06941v2 Announce Type: replace-cross Abstract: Large language models can recover mid-generation from task-misaligned activation steering, producing explicit verbal restarts (e.
arXiv:2609.35932v1 Announce Type: cross Abstract: Prompt injection against LLM agents becomes much stronger when the injected instruction is wrapped in the model's own chat template. A forged templat...
arXiv:2606. 29441v1 Announce Type: cross Abstract: Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists.
arXiv:2609. 31181v1 Announce Type: new Abstract: Black-box model identification works by scoring a model's response to natural-language prompts.
arXiv:2609.00064v1 Announce Type: cross Abstract: In-context learning (ICL) lets large language models adapt to new tasks from demonstrations, and fine-tuning can erode this behaviour. Many preservat...
arXiv:2606. 07889v1 Announce Type: cross Abstract: LLM-based coding agents sometimes acknowledge a problem in their own reasoning and then proceed anyway.