The paper introduces the Latent Diagnostic Taxonomy, a framework that builds a dimensionality‑optimized classifier and a diagnostic tool to assess the trustworthiness of its confident predictions. It identifies a small set of influential prompts (latent support vectors) that reveal tokens which can change the classifier’s output, and uses these tokens to create a taxonomy that classifies prompts into safe, heuristic bias, heuristic override, or insufficient context categories. Applied to a prompt‑injection detection model, the framework shows that about 77% of confident decisions are fragile to a single token, distinguishing between calibration failures and exploitable shortcuts, and offers remediation strategies for each taxonomy zone.
This paper proposes a framework for constructing a classifier as a safeguard layer, and for developing a complementary diagnostic that identifies which of the classifier's confident decisions can be t...
arXiv:2602. 14161v2 Announce Type: replace Abstract: Detecting prompt injection, jailbreak attacks, and harmful requests is critical for deploying LLM-based agents safely, yet current evaluation practices in this literature overestimate generalization.
arXiv:2609.15017v1 Announce Type: cross Abstract: Prompt-injection detectors are typically evaluated using aggregate F1 on in-distribution test data, which offers limited insight into behavior under...
The paper introduces a prompt‑injection detection framework for email assistants that models attacks as a chain of stages. It combines a text detector, stage‑specific verifiers, rule‑based risk signals, user intent consistency checks, and a logistic decision policy. Experiments on five benchmarks show the framework outperforms pretrained detectors, achieving a mean F1 of 0.406 versus 0.216, and demonstrate that training on benign emails resembling attacks reduces false alarms.
arXiv:2607. 28460v1 Announce Type: new Abstract: A major issue in Security Operations Centers (SOCs) is alert fatigue, as the number of detections reported is more than staff can triage in a given day.
The paper evaluates TypeSafe AI’s Jev safety judge against GPT and GLM judges and local classifiers using a controlled adversarial test harness. Jev shows strong robustness on rule‑based tasks like input moderation, with minimal bypasses, but performs less well on judgment‑based tasks such as phishing detection, where it misses a higher proportion of phishing emails and is more prone to false positives. The study highlights Jev’s speed advantage and varying resilience depending on task type.
The paper introduces DiagSafe, a diagnostic evaluation framework that replaces binary safe/unsafe labels with prompt-level rubrics grounded in AI safety policies. DiagSafe uses a bilingual benchmark with an 84‑label safety taxonomy, comprising DiagBase and DiagAttack subsets, to assess safety and usefulness independently. Experiments show DiagSafe achieves high agreement with human annotations (95.34%) and strong inter‑judge consistency (Cohen’s κ = 0.864), revealing significant differences in unsafe rates and vulnerabilities across leading LLMs, especially in Chinese and multi‑turn contexts.
arXiv:2606. 27091v1 Announce Type: cross Abstract: LLMs fine-tuned for security classification are usually evaluated on held-out examples from the same distribution as their training data.
A major issue in Security Operations Centers (SOCs) is alert fatigue, as the number of detections reported is more than staff can triage in a given day. Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat.
arXiv:2605. 23055v2 Announce Type: replace-cross Abstract: Frontier language models sometimes recognize that they are being evaluated and adjust their behavior, undermining validity of benchmark results.
CoGReV is a hybrid framework that enhances machine‑learning phishing classifiers with a post‑hoc, non‑monotonic reasoning layer written in Answer Set Programming. It uses a confidence‑gated defeasible rule to revise low‑confidence phishing predictions toward legitimate only when website metadata is available, thereby allocating uncertain decisions to the reasoning layer while leaving confident ones to the classifier. The gated rule reduces false positives by 0.27 % of decisions and maintains recall within 0.7 % of the baseline, operating in linear time.
arXiv:2602.06268v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly integrated into clinical workflows. However, p...