arXiv AI
Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Sep 4
Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models
The paper introduces a new measure of generative‑process diversity for language models, using Normalised Compression Distance on raw outputs after controlling for permutation effects. Across 38 models, this metric uncovers population structure that semantic similarity misses and predicts lower correlated failures across ten benchmark families, independent of semantic similarity or model capability. The authors argue that higher generative‑process diversity reduces correlated failures in multi‑model systems, offering a practical tool for safety‑relevant applications.
Hugging Face Trending Papers
Sep 3
Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models
The paper argues that traditional semantic similarity fails to capture the true diversity of language models. It introduces a new metric—generative‑process diversity—measured via Normalised Compression Distance on raw outputs, which reveals hidden population structure among 38 models. This metric predicts lower correlated failures across ten benchmark families, independent of semantic similarity or model capability.
arXiv AI
Aug 24
Lost in Sampling: Assessing Lexical Reachability in LLMs via the Word Coverage Score (WCS)
arXiv:2605.27268v2 Announce Type: replace-cross Abstract: Modern Large Language Models (LLMs) are often criticized for producing repetitive and homogeneous text, despite possessing vast latent vocabu...
arXiv Machine Learning
Aug 11
On the Effect of Sampling Diversity in Scaling LLM Inference
arXiv:2502. 11027v5 Announce Type: replace Abstract: Large language model (LLM) scaling inference is key to unlocking greater performance, and leveraging diversity has proven an effective way to enhance it.
arXiv AI
3d ago
LLM Probability Concentration: How Alignment Shrinks the Generative Horizon
arXiv:2506.17871v4 Announce Type: replace-cross Abstract: Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this cons...
arXiv AI
Jun 30
Using Large Language Models as Low-Cost Statistical Estimators for Human-Response Data
arXiv:2606. 30372v1 Announce Type: new Abstract: Quantitative research across the social and behavioral sciences depends on human subject experiments that are expensive, slow, and subject to sampling bias.