arXiv AI
Jul 24
Refusal-Gated Decoding: Preserving Refusal Behavior Under High-Temperature Sampling
arXiv:2607. 20791v1 Announce Type: new Abstract: High-temperature sampling is one of the primary mechanisms for increasing diversity in LLMs.
Temperature Fragility and the Conditional Benefits of Truncation Sampling
Related stories
arXiv AI
Jun 30
The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis
arXiv:2606. 29581v1 Announce Type: cross Abstract: Modern LLM deployments routinely compress models and raise sampling temperature to reduce cost, latency, or repetition, yet safety evaluations usually treat these choices as fixed implementation details.
arXiv Machine Learning
5d ago
Less Loss, Fixed Weights: Logit Corrections for Forgetting
The paper investigates whether a language model’s forgetting—worsening performance on standard web text after fine‑tuning or further pretraining—can be mitigated by simple, fixed adjustments to its final token predictions. By fitting a vocabulary‑wide bias (a token‑specific bonus or penalty) and optionally a temperature scaling, the authors evaluate recovery across 14 base‑to‑derived model pairs ranging from 1B to 7B parameters. Results show modest gains: up to 10% recovery with bias alone and up to 36% when combined with temperature, though models further pretrained on math lose the most and recover little; even in a controlled simulation with random damage, the bias‑plus‑temperature fix recovers 29%.
arXiv Machine Learning
Sep 30
Cool the Sampler, Not the Learner: Sampling Temperature Moves the Staleness Cliff of Importance-Corrected GRPO
arXiv:2609.36953v1 Announce Type: new Abstract: Production RL for language models lets the sampler fall behind the learner and repairs the resulting mismatch with a truncated importance weight. We as...
Hugging Face Trending Papers
Jun 28
The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis
Modern LLM deployments routinely compress models and raise sampling temperature to reduce cost, latency, or repetition, yet safety evaluations usually treat these choices as fixed implementation details. This leaves a practical uncertainty: does a model that is safe at FP16 and greedy decoding remain safe after it is quantized and sampled stochastically, or do the two deployment knobs amplify one another?
arXiv Machine Learning
Sep 23
Modular Norm RandOpt: Population-Efficient Ensembling through Architecture-Aware Perturbations
arXiv:2609.25745v1 Announce Type: new Abstract: RandOpt samples weight-perturbed language models and ensembles top-ranked candidates through plurality voting, but its global perturbation scale ignore...
arXiv AI
Sep 2
The Importance of Being Statistically Earnest: A Critical Re-evaluation of GSM-Symbolic
The paper re‑examines the GSM‑Symbolic benchmark, arguing that its claim of widespread reasoning failures in 25 LLMs is based on weak statistics. Using bootstrapped Generalised Linear Mixed Models on 20 open‑weight models, only eight show significant performance changes, and a systematic shift toward larger integers in the dataset explains many of these effects. The authors also uncover model‑specific failure modes such as variable binding fragility, arithmetic limits, and dual‑task interference, cautioning against blanket conclusions about LLM reasoning.
arXiv Machine Learning
Aug 31
The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior
The paper challenges the assumption that large language models (LLMs) produce deterministic safety responses by examining how random seeds and temperature settings affect refusal decisions. Across four instruction‑tuned models and 876 harmful prompts, 18‑28% of prompts flipped between refusal and compliance depending on sampling configuration, with higher temperatures reducing decision stability. The authors introduce a Safety Stability Index (SSI) and recommend multi‑sample evaluation protocols that account for stochastic variation rather than relying on single‑shot tests.
arXiv Machine Learning
Sep 16
Bridging the Confidence Gap: Temperature Scaling for Calibrating Test-Time Prompt Tuning
arXiv:2609.17386v1 Announce Type: new Abstract: Test-time prompt tuning (TPT) enables adaptation on a single test instance, achieving improved accuracy but often sacrificing calibration performance....
arXiv AI
Oct 15
Lightweight but Linguistically Fair? Quantisation Robustness across Bavarian, German, and English
arXiv:2610.18343v1 Announce Type: cross Abstract: Post-training quantisation reduces the memory cost of large language models, but it remains unclear whether compression preserves language skills of...
arXiv AI
Aug 6
Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models
arXiv:2608. 05064v1 Announce Type: cross Abstract: Small open-weight language models increasingly run in private, offline, and cost-sensitive settings, where the key deployment question is not only what a model answers but when it should defer to a human.
arXiv Machine Learning
Aug 4
Conformalized Large Language Models under Configuration Shift
arXiv:2608. 01460v1 Announce Type: new Abstract: Conformal prediction (CP) is a distribution-free framework for uncertainty quantification that has recently been adapted to large language models (LLMs), providing prediction sets with finite-sample coverage guarantees under exchangeability.