arXiv:2607. 20286v1 Announce Type: cross Abstract: We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt.
By Mahdi Nazeri, Anne-Kathrin Schmuck, Sadegh Soudjani, Alessandro Abate
arXiv:2604. 22167v2 Announce Type: replace-cross Abstract: Language models are increasingly capable and are being rapidly deployed on a population-level scale.
By Rico Angell, Raghav Singhal, Zachary Horvitz, Zhou Yu, Rajesh Ranganath, Kathleen McKeown, He He
arXiv:2608. 10621v1 Announce Type: new Abstract: Recent research on Large Language Model (LLM) safety has widely adopted guardrails to identify unsafe LLM outputs.
By Xinzhe Huang, Biwu Yao, Kedong Xiu, Mengnan Zhao, Di Wang, Puning Zhao, Tianhang Zheng
arXiv:2608. 15798v1 Announce Type: new Abstract: Language models are compared by their held-out per-token cross-entropy risk---the quantity scaling laws are fitted to.
By Hanti Lin
arXiv:2607. 19367v1 Announce Type: new Abstract: Calibration is the primary criterion for evaluating LLM confidence, but it is insufficient: it admits trivially incoherent estimators, depends on the evaluation distribution, and does not test the extent to which the estimation can be interpreted as a consistent, underlying probability function.
By Krish Matta, Atharv Naphade, Andy Zou
The paper introduces Word-level Probability MIA (WPMIA), a black-box membership inference attack that estimates word-level generation probabilities via Monte Carlo sampling and local kernel smoothing, then aggregates them into a sequence-level likelihood estimator. By conditioning on different prefixes, WPMIA amplifies distributional differences between member and non-member texts, outperforming existing black-box baselines on open-source LLMs and achieving an average TPR@5%FPR of 42.0 on proprietary models such as GPT‑5‑Chat, Gemini‑2.5‑Flash, and Claude‑4.5‑Haiku.
By Shengjie Niu, Yeheng Ge, Jian Huang