arXiv AI
Coherence-Aware Distributional Evaluation of Open-Ended Text Generation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Sep 24
Reference-Based Analysis of Coherence and Diversity in Open-Ended Text Generation
The paper introduces a reference-based framework to analyze coherence and diversity in open-ended text generation. It evaluates these properties by aligning them with human trajectories, comparing them to human continuations, and estimating their likelihood under a human reference distribution. Experiments show that diversity alignment and mean-based comparisons correlate with human quality ratings, while reference likelihood also associates positively, though results vary by configuration.
arXiv AI
Jul 8
RPAM: A Principled Metric for Evaluating Associations in Language Models with High Predictive Validity in Downstream Outputs
arXiv:2607. 05679v1 Announce Type: cross Abstract: Language models (LMs) exhibit problematic biases, such as stereotypes.
arXiv AI
Jul 7
Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)
arXiv:2607. 04223v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) reduces but does not eliminate hallucination, and existing detectors return a single answer-level score that does not indicate which sentence is unsupported, or why.
arXiv AI
Jun 9
Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics
arXiv:2606. 08417v1 Announce Type: cross Abstract: Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling.
arXiv AI
Jun 11
Geometric Metrics and LLMs: What They Measure and When They Work
arXiv:2509. 25359v2 Announce Type: replace-cross Abstract: We present a systematic stress-test of geometric metrics for LLM evaluation.
arXiv AI
Sep 15
Context-Dependent Affordance Reports in Vision-Language Models
arXiv:2603.04419v3 Announce Type: replace-cross Abstract: Vision-language models produce different object and use descriptions under different persona prompts, but low overlap alone does not identify...