The paper investigates why Transformers struggle more with structural than lexical compositional generalisation. It argues that this disparity stems from low structural type diversity rather than an inherent limitation of Transformers. By creating linguistically diverse variants of the COGS and SLOG datasets, the authors show that type diversity correlates equally with generalisation in both lexical and structural cases, challenging previous explanations of the difficulty.
By Anssi Moisio, Mathias Creutz, Mikko Kurimo
arXiv:2606. 01811v1 Announce Type: cross Abstract: Measuring the diversity of creative outputs is central to evaluating post-training mode collapse, comparing decoding strategies, and quantifying creative behavior in both AI and human writing.
By Matthew Khoriaty, David Williams-King, Shi Feng
arXiv:2605.27268v2 Announce Type: replace-cross
Abstract: Modern Large Language Models (LLMs) are often criticized for producing repetitive and homogeneous text, despite possessing vast latent vocabu...
By Samer Awad, Javier Conde, Carlos Arriaga, Tairan Fu, Javier Coronado-Bl\'azquez, Pedro Reviriego
The paper argues that traditional semantic similarity fails to capture the true diversity of language models. It introduces a new metric—generative‑process diversity—measured via Normalised Compression Distance on raw outputs, which reveals hidden population structure among 38 models. This metric predicts lower correlated failures across ten benchmark families, independent of semantic similarity or model capability.
The paper introduces a new measure of generative‑process diversity for language models, using Normalised Compression Distance on raw outputs after controlling for permutation effects. Across 38 models, this metric uncovers population structure that semantic similarity misses and predicts lower correlated failures across ten benchmark families, independent of semantic similarity or model capability. The authors argue that higher generative‑process diversity reduces correlated failures in multi‑model systems, offering a practical tool for safety‑relevant applications.
By Ross Tieman, Evan Markou
The paper argues that measuring diversity in AI-generated content using a single scalar score is inherently ambiguous and often misleading. It reviews existing diversity metrics, demonstrates their limitations through axiomatic and empirical analyses, and introduces diversity profiles—curve-valued, condition-aware summaries that evaluate diversity across a range of thresholds, scales, exponents, or orders. These profiles reveal whether comparisons are robust across resolutions or depend on arbitrary parameter choices, offering a more transparent framework for generative AI evaluation.
By Xiuyuan Hu, Xuege Hou, Guoqing Liu, Yang Zhao, Jieran Li, Dongbiao Sun, Jos\'e Miguel Hern\'andez-Lobato, Hao Zhang, Xue Liu
The paper proposes a depth‑aware FrankenMerging technique that blends lower layers of an aligned large language model (LLM) with higher layers of its base counterpart to balance diversity and quality without extra training. Experiments demonstrate that adjusting the layer split controls both lexical and semantic diversity, often surpassing the aligned model in semantic diversity and outperforming temperature sampling in most settings. The method is validated by an ablation that reverses the merge order, supporting the hypothesis about layer‑specific diversity contributions.
By Yasuaki Sumita, Tomoharu Iwata, Toshiyuki Tanaka
arXiv:2502. 11027v5 Announce Type: replace Abstract: Large language model (LLM) scaling inference is key to unlocking greater performance, and leveraging diversity has proven an effective way to enhance it.
By Tianchun Wang, Zichuan Liu, Yuanzhou Chen, Jonathan Light, Weiyang Liu, Haifeng Chen, Xiang Zhang, Wei Cheng
arXiv:2512. 15011v3 Announce Type: replace Abstract: Artificial intelligence (AI) increasingly generates the very content used to train future AI systems.
By Damian Hodel, Jevin D. West
arXiv:2610.14031v1 Announce Type: cross
Abstract: Modern data pipelines increasingly combine human-created and human-curated data with AI-generated data for model training and evaluation. As datasets...
By Yang Ba, Michelle V. Mancenido, Rong Pan
arXiv:2602. 00238v2 Announce Type: replace-cross Abstract: Existing retrieval-augmented generation (RAG) systems often assume that each query has a single correct answer.
By Tianyi Hu, Niket Tandon, Akhil Arora
The study evaluates how sixteen language models from ten families generate diverse formulations of a psychotherapeutic case, finding an average semantic diversity of 1.69 distinct formulations versus 1.43 for a single-model baseline. It introduces the Vendi Score to quantify diversity and defines a per-model dissent metric to identify the most divergent voice within an ensemble. The analysis shows that model identity significantly influences dissent, but this effect varies across model pairs and panel compositions, indicating that ensemble dispersion is a measurable property rather than an assumed one.
By Mario Vega-Barbas, Lidia Mora-Valenciano, Iv\'an Pau, Fernando Seoane, Farhad Abtahi