Hugging Face Trending Papers
XTC: Head-Aware Sampling by Excluding Top Choices
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Aug 25
XTC: Head-Aware Sampling by Excluding Top Choices
XTC (Exclude Top Choices) is a lightweight, head‑aware decoding operator that improves diversity in autoregressive language models by removing overly probable tokens that dominate the next‑token distribution. It works by identifying tokens above a plausibility threshold, probabilistically excluding the dominant choices, and renormalizing the remaining distribution. Across 60 experiments on models such as Gemma 3 and DeepSeek R1, XTC boosts Distinct‑2 scores by 11–15 % and cuts repeat trigrams by 27–47 %, while a Mechanical Turk study shows a 62.3 % preference for XTC‑generated text without loss of fluency.
arXiv AI
Sep 10
Quantization Amplifies Determinism, Not Bias: Scale-Dependent Behavioral Effects of Serving-Time Weight Compression
arXiv:2609.07901v1 Announce Type: new Abstract: Weight quantization largely determines the economics of serving open-weight LLMs. Its costs are usually assessed with capability benchmarks, on which 4...
arXiv Computation and Language
Sep 4
Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding
The paper introduces Variance‑Calibrated Modulation (VCM), a training‑free pre‑decoding technique that reshapes language model probability distributions before truncation. VCM uses two dynamic mechanisms: a Contextual Searchlight via PMI to suppress stopwords and highlight context‑relevant tokens, and an Adaptive Self‑Debiasing that applies scale‑invariant penalization based on real‑time logit standard deviation. Experiments on open‑ended generation, factual QA, and mathematical reasoning show that VCM consistently reduces the likelihood trap, improving diversity, coherence, and reasoning accuracy with minimal computational cost.
arXiv Computation and Language
Oct 1
Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models
arXiv:2410.02343v2 Announce Type: replace Abstract: Large language models (LLMs) routinely fail to output the correct option in multiple-choice question answering (MCQA) while encoding the answer int...
arXiv AI
Jun 2
S-SPPO: Semantic-Calibrated Self-Play Preference Optimization
arXiv:2606. 01561v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) with human preferences is often formulated via Direct Preference Optimization (DPO).
Hugging Face Trending Papers
Oct 29
Neighborhood Elites: Quality-Diversity Search for Uncertainty Quantification in Question Answering
The paper introduces Neighborhood Elites (NE), a quality‑diversity search method that constructs input rewrites for large language models in question answering by controlling both the type of edit and the explicitness of support. NE outperforms three other neighborhood constructors across five extractive‑QA datasets and five instruction‑tuned LLMs, achieving higher AUROC and AUPRC scores and surpassing semantic entropy from decoder samples. The method demonstrates that carefully designed perturbations improve failure detection in QA systems.