← Back to all news
arXiv Machine Learning By Gregor Kornhardt, Moritz Piening, Jannis Chemseddine, Gabriele Steidl

SOL: Measuring Gaps between Text Distributions by Double Sliced Wasserstein Metrics

Read the original on arXiv Machine Learning →

The paper introduces SOL, a new metric for measuring gaps between text distributions using double sliced Wasserstein distances. It represents each sequence as an empirical measure of hidden states from a fixed transformer and compares these measures across distributions. The authors prove SOL is a metric when the transformer is injective and demonstrate its effectiveness in detecting distributional failures and recovering expected model trends in experiments on OpenWebText models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
Oct 23

Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs New Metrics

The paper critiques generative perplexity (gen‑PPL) as an unreliable metric for evaluating unconditional text generation, showing that it can be minimized by trivial, non‑diverse outputs. The authors demonstrate this by constructing simple samplers that achieve low gen‑PPL yet produce incoherent text, and they propose alternative evaluation tools—including distributional discrepancies, diversity diagnostics, and a compression‑based KL divergence measure—to better assess generative language models. They argue that future progress in diffusion language models should rely on a suite of metrics that capture diverse aspects of generated text quality.

By Antonio Franca, Alexander Tong
llmsdiffusion
More like this →
arXiv Machine Learning
Oct 16

Vertical Sliced Wasserstein Distances for Meta-Measures

arXiv:2610. 20743v1 Announce Type: new Abstract: Many modern learning problems require comparing probability distributions whose individual observations are themselves probability measures, naturally giving rise to meta-measures in $\mathcal{P}(\mathcal{P}(\Omega))$.

By Elaheh Akbari, Ahmadreza Moradipari, Kyungtae Han, Hamed Pirsiavash, Yikun Bai, Rocio Diaz Martin, Soheil Kolouri
More like this →