Modern LLM deployments routinely compress models and raise sampling temperature to reduce cost, latency, or repetition, yet safety evaluations usually treat these choices as fixed implementation details. This leaves a practical uncertainty: does a model that is safe at FP16 and greedy decoding remain safe after it is quantized and sampled stochastically, or do the two deployment knobs amplify one another?
The paper challenges the assumption that large language models (LLMs) produce deterministic safety responses by examining how random seeds and temperature settings affect refusal decisions. Across four instruction‑tuned models and 876 harmful prompts, 18‑28% of prompts flipped between refusal and compliance depending on sampling configuration, with higher temperatures reducing decision stability. The authors introduce a Safety Stability Index (SSI) and recommend multi‑sample evaluation protocols that account for stochastic variation rather than relying on single‑shot tests.
By Erik Larsen
arXiv:2606. 09864v1 Announce Type: cross Abstract: Key-value (KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory, yet existing evaluations solely focus on measuring perplexity and accuracy without assessing the safety impact.
By Bruce Changlong Xu, Adarsh Kumarappan, Mu Zhou
arXiv:2606. 25097v1 Announce Type: new Abstract: Speculative decoding accelerates inference by letting a draft model propose tokens for a target model to verify, raising a concrete safety question: at temperature zero, can draft-side behavior leak into safety-scored outputs?
By Sahil Kadadekar
The paper investigates where post‑training quantization (PTQ) harms large language models (LLMs) and how to best allocate a limited precision budget. By causally raising each layer to 8‑bit precision across nine open‑weight models, the authors find that quantization damage is diffuse rather than concentrated in specific task circuits or weight statistics, and that globally refining quantization granularity outperforms selectively protecting the most recoverable layers. They also observe that the residual accuracy loss is budget‑limited and that peak recovery locations correlate with architecture within families but not across families.
By Jundong Hu, Shekar Ramachandran
arXiv:2608. 06564v1 Announce Type: new Abstract: Quantization is how large language models are actually deployed, and below four bits it is known to hurt.
By Zekun Wu, Swati Dhiman, Adriano Koshiyama