The paper introduces Jev, a reinforcement‑learning‑trained model that provides calibrated probability answers to typed questions about a single input in one call. Jev is evaluated on RLCDAlignBench, a benchmark covering ten alignment failures across 44 tests and five target models, achieving a median AUROC of 0.886 zero‑shot and outperforming supervised baselines on most tasks. The study shows that question wording has little impact, while contextual fields that encode labels are more influential, and that Jev matches human‑label agreement while being 63× cheaper than LLM‑judge scorers.
By Ruoqi Guo, Yi Liu, Gelei Deng, Yuekang Li, Lida Zhao, Yutao Wu, Simin Chen, Ying Zhang, Leo Yu Zhang
arXiv:2609.36570v1 Announce Type: cross
Abstract: Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that...
By Mark Russinovich
The paper evaluates TypeSafe AI’s Jev safety judge against GPT and GLM judges and local classifiers using a controlled adversarial test harness. Jev shows strong robustness on rule‑based tasks like input moderation, with minimal bypasses, but performs less well on judgment‑based tasks such as phishing detection, where it misses a higher proportion of phishing emails and is more prone to false positives. The study highlights Jev’s speed advantage and varying resilience depending on task type.
By Xiang Li, Aichun Huang, Mingming Zhang, Zheli Liu
This thesis introduces methods for assessing and enhancing the robustness of large language models (LLMs) against adversarial input variations. It proposes a generative robustness metric, R_stab(f), based on Jensen‑Shannon divergence, and proves bounds for localized attacks. The work also presents adaptive attacks, Trojan detection techniques, defense strategies for multi‑layer systems, and lightweight attestation for Model Context Protocol (MCP) agentic systems, all implemented in the JudgeGuard, TrojanArmor, and MCPSec suites.
By Narek Maloyan
arXiv:2609.38850v1 Announce Type: new
Abstract: Decision models such as Jev answer questions with probabilities, which are only useful if they are calibrated. Open-source reproductions rely on superv...
By Zhimin Gao, Pichao Wang
arXiv:2607. 05904v1 Announce Type: new Abstract: Training a language model against its own reference-free judgments (the premise of self-rewarding, self-play, and LLM-as-a-judge pipelines) assumes a model's verdict on a shown answer tracks correctness.
By Chenyu Zhou