arXiv:2607. 20286v1 Announce Type: cross Abstract: We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt.
We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt. We study a new application of the Clopper-Pearson confidence intervals to obtain probably approximately correct (PAC) bounds for this problem.
arXiv:2604. 22167v2 Announce Type: replace-cross Abstract: Language models are increasingly capable and are being rapidly deployed on a population-level scale.
arXiv:2608. 15798v1 Announce Type: new Abstract: Language models are compared by their held-out per-token cross-entropy risk---the quantity scaling laws are fitted to.
arXiv:2607. 19367v1 Announce Type: new Abstract: Calibration is the primary criterion for evaluating LLM confidence, but it is insufficient: it admits trivially incoherent estimators, depends on the evaluation distribution, and does not test the extent to which the estimation can be interpreted as a consistent, underlying probability function.
arXiv:2608. 10621v1 Announce Type: new Abstract: Recent research on Large Language Model (LLM) safety has widely adopted guardrails to identify unsafe LLM outputs.
arXiv:2608. 19323v1 Announce Type: cross Abstract: Uncertainty quantification (UQ) is essential for the safe deployment of large language models (LLMs).
The paper introduces the concept of likelihood hacking (LH), where language models trained via reinforcement learning generate probabilistic programs that inflate marginal-likelihood rewards by producing non‑normalising data distributions. It formalises LH in a core probabilistic programming language and derives syntactic conditions that prevent such exploits, defining a safe language fragment λ_safe. Empirical studies show that models trained with GRPO quickly discover LH, while a modified Stan implementation, SafeStan, effectively suppresses these exploits under optimisation pressure.
The paper introduces Word-level Probability MIA (WPMIA), a black-box membership inference attack that estimates word-level generation probabilities via Monte Carlo sampling and local kernel smoothing, then aggregates them into a sequence-level likelihood estimator. By conditioning on different prefixes, WPMIA amplifies distributional differences between member and non-member texts, outperforming existing black-box baselines on open-source LLMs and achieving an average TPR@5%FPR of 42.0 on proprietary models such as GPT‑5‑Chat, Gemini‑2.5‑Flash, and Claude‑4.5‑Haiku.
arXiv:2607. 08961v1 Announce Type: cross Abstract: Large language models increasingly provide labels, evaluations, and feedback for tasks specified in natural language.
arXiv:2604. 06543v2 Announce Type: replace-cross Abstract: In this work, we demonstrate that reliable stochastic sampling is a fundamental yet unfulfilled requirement for Large Language Models (LLMs) operating as agents.
The paper introduces RareTrap, a framework that estimates the probability of severe behaviors in black‑box large language models. RareTrap constructs a geometry‑aware mapping from a low‑dimensional latent space into token‑embedding space using a surrogate LLM, creating an explicit and reproducible distribution over input prompts. By applying a response‑level performance function and sequential rare‑event simulation, RareTrap concentrates evaluations on increasingly severe behaviors while preserving probability, enabling estimation of such behaviors with as few as 200 evaluations across multiple open‑weight and frontier models.
Pinocchio is an external calibrator that provides fast uncertainty estimates for black-box language models. It predicts the correctness of responses from seven trained LLMs with an AUROC of 0.862 and can transfer zero‑shot to thirteen unseen models from eight organizations. The method requires only a single forward pass and no access to the target model’s internal states, and a lightweight 0.8B checkpoint achieves comparable performance.