The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper demonstrates that Matryoshka Representation Learning (MRL) embeddings contain a natural coarse‑to‑fine hierarchy, with prefix length aligning to gold taxonomy granularity. By training a simple order‑embedding probe called PEEL on these frozen embeddings, the authors can recover high‑fidelity taxonomic DAGs from a concept vocabulary alone, achieving strong results on two OLLM benchmarks and transferring to new domains without target‑edge supervision. The study shows that pre‑trained retrieval embeddings encode a decodable granularity gradient, enabling a minimal probe to extract explicit parent‑child (is‑a) structure.
Granuscore is a reference‑free metric that measures the granularity of text by exploiting the structure of a hierarchical embedding space. It successfully reproduces known hierarchical orderings on the Granola‑EQ dataset, distinguishes granularity across different discourse contexts, and explains sentence‑specificity variations beyond sentence length. The authors also apply Granuscore to four question‑answering benchmarks, revealing systematic differences in granularity among questions, gold answers, and model outputs, thereby offering a new lens for assessing QA dataset difficulty.
arXiv:2608. 07208v1 Announce Type: cross Abstract: Existing measures of how much a text is about a concept read the surface of the text: dictionary word shares, topic proportions, embedding similarities.
TACTICS is a method for selecting evaluation samples in machine translation that explicitly optimizes for coverage of rare linguistic categories, document-level coherence, and distributional fidelity to the full corpus. It builds a hierarchical taxonomy from a locale style guide, classifies segments, and chooses a fixed-budget subset that better represents the full range of phenomena a system must handle. Compared to random, lexical, or embedding-based selection, TACTICS improves coverage of rare categories and yields more accurate system rankings with fewer segments.
The paper introduces a taxonomy for the term "AI slop," a label for low‑quality AI‑generated text, and proposes interpretable dimensions to assess it. Using expert interviews and span‑level annotation, the authors find that binary slop judgments are somewhat subjective yet correlate with latent factors like coherence and relevance. Their framework can be applied to both detection and preference tasks, offering insights into linguistic and stylistic contributors to quality, while noting that fully automated, scalable methods remain an open challenge.
arXiv:2608. 10970v1 Announce Type: cross Abstract: Recent advances in Large Language Models (LLMs) have demonstrated strong capabilities in generating semantically relevant concepts and relations, making them promising tools for taxonomy enrichment.