arXiv AI
PLDiv: Measuring Dataset Diversity and Semantic Collapse via Persistence Landscapes
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 19
Evaluating the Diversity of AI-Generated Content with Diversity Profiles
The paper argues that measuring diversity in AI-generated content using a single scalar score is inherently ambiguous and often misleading. It reviews existing diversity metrics, demonstrates their limitations through axiomatic and empirical analyses, and introduces diversity profiles—curve-valued, condition-aware summaries that evaluate diversity across a range of thresholds, scales, exponents, or orders. These profiles reveal whether comparisons are robust across resolutions or depend on arbitrary parameter choices, offering a more transparent framework for generative AI evaluation.
arXiv Computer Vision
Aug 25
Mapping the Concept Landscape: Structural Perception of Global Distributions for Transparent Data Pruning
The paper introduces Mapping the Concept Landscape (MCL), a framework that replaces high‑dimensional feature embeddings with explicit sample‑level graphs of entities, events, and attributes for image‑caption pairs. By aggregating these graphs into a dataset‑level graph, MCL captures the global distribution of semantic concepts and identifies rare concepts. A greedy algorithm then selects samples to maximize coverage of under‑represented concepts, achieving better pruning efficiency and providing a transparent audit trail.
arXiv AI
Jul 28
CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data
arXiv:2607. 22662v1 Announce Type: new Abstract: Open-web corpora curated via highly selective filters, such as FineWeb-Edu and DCLM, constitute the core of LLM pretraining data and have significantly advanced LLM performance.
Hugging Face Trending Papers
Jun 22
Semantic Browsing: Controllable Diversity for Image Generation
Modern text-to-image models excel in visual fidelity and prompt adherence. However, this strict adherence comes at the cost of diversity: generated samples tend to collapse into a single visual interpretation.
arXiv AI
Jul 21
Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
arXiv:2505. 20161v2 Announce Type: replace-cross Abstract: Effective generalization in language models depends critically on the diversity of their training data.
arXiv AI
Sep 10
ECOKV: Geometry-Aware KV Cache Eviction via Complementary Diversity Metrics
arXiv:2609.06663v1 Announce Type: cross Abstract: Although multimodal Large Language Models (MLLMs) excel in diverse tasks, their scalability remains limited by the memory and computational overhead...