The paper demonstrates that Matryoshka Representation Learning (MRL) embeddings contain a natural coarse‑to‑fine hierarchy, with prefix length aligning to gold taxonomy granularity. By training a simple order‑embedding probe called PEEL on these frozen embeddings, the authors can recover high‑fidelity taxonomic DAGs from a concept vocabulary alone, achieving strong results on two OLLM benchmarks and transferring to new domains without target‑edge supervision. The study shows that pre‑trained retrieval embeddings encode a decodable granularity gradient, enabling a minimal probe to extract explicit parent‑child (is‑a) structure.
By Eunchan Kim, Inwhee Joe
Granuscore is a reference‑free metric that measures the granularity of text by exploiting the structure of a hierarchical embedding space. It successfully reproduces known hierarchical orderings on the Granola‑EQ dataset, distinguishes granularity across different discourse contexts, and explains sentence‑specificity variations beyond sentence length. The authors also apply Granuscore to four question‑answering benchmarks, revealing systematic differences in granularity among questions, gold answers, and model outputs, thereby offering a new lens for assessing QA dataset difficulty.
By Lukas Ellinger, Alexander Fichtl, Miriam Ansch\"utz, Georg Groh
arXiv:2608. 07208v1 Announce Type: cross Abstract: Existing measures of how much a text is about a concept read the surface of the text: dictionary word shares, topic proportions, embedding similarities.
By Luc Hazenoot, Zhaochun Ren, Amirhossein Zohrehvand
TACTICS is a method for selecting evaluation samples in machine translation that explicitly optimizes for coverage of rare linguistic categories, document-level coherence, and distributional fidelity to the full corpus. It builds a hierarchical taxonomy from a locale style guide, classifies segments, and chooses a fixed-budget subset that better represents the full range of phenomena a system must handle. Compared to random, lexical, or embedding-based selection, TACTICS improves coverage of rare categories and yields more accurate system rankings with fewer segments.
By Prasanth Bathala, Anubhav Shrimal, Sukhdeep Singh Kharbhanda, Pradyumna Lanka, Rohit Dhaipule
The paper introduces a taxonomy for the term "AI slop," a label for low‑quality AI‑generated text, and proposes interpretable dimensions to assess it. Using expert interviews and span‑level annotation, the authors find that binary slop judgments are somewhat subjective yet correlate with latent factors like coherence and relevance. Their framework can be applied to both detection and preference tasks, offering insights into linguistic and stylistic contributors to quality, while noting that fully automated, scalable methods remain an open challenge.
By Chantal Shaib, Tuhin Chakrabarty, Diego Garcia-Olano, Byron C. Wallace
arXiv:2608. 10970v1 Announce Type: cross Abstract: Recent advances in Large Language Models (LLMs) have demonstrated strong capabilities in generating semantically relevant concepts and relations, making them promising tools for taxonomy enrichment.
By Zeinab Ghamlouch, Mehwish Alam
Recent advances in Large Language Models (LLMs) have demonstrated strong capabilities in generating semantically relevant concepts and relations, making them promising tools for taxonomy enrichment. However, directly relying on LLM-generated expansions often leads to noisy, redundant, or hierarchically inconsistent structures, limiting their reliability for automated taxonomy expansion.
arXiv:2604. 09497v2 Announce Type: replace-cross Abstract: Accurate evaluation is central to the large language model (LLM) ecosystem, guiding model selection and downstream adoption across diverse use cases.
By Hippolyte Gisserot-Boukhlef, Nicolas Boizard, Emmanuel Malherbe, C\'eline Hudelot, Pierre Colombo
The paper introduces the Concept Field, a local drift field with pointwise uncertainty derived from sentence-embedding deltas in a text corpus. It scores candidate sentence transitions using the mean absolute z-distance (ζ) to quantify groundedness, offering a model‑agnostic, corpus‑attributable metric. The authors implement a Vector Sequence Database to store embeddings and delta metadata, and demonstrate the method’s effectiveness in detecting groundedness in the U.S. Code of Federal Regulations and novelty in Project Gutenberg, achieving strong selective classification performance under a triage policy.
By Nicholas S. Kersting, Vittorio Castelli, Chieh Ting Yeh, Xinzhu Wang, Saad Taame, Khaoula Allak
arXiv:2610.09063v1 Announce Type: new
Abstract: Multi-label topic assignment for user-generated content (UGC) -- including product reviews and buyer-seller conversations -- poses unique scalability c...
By Sourabh Kasliwal, Shubhranshu Singh
arXiv:2606. 27226v1 Announce Type: new Abstract: Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores that are hard to debug.
By Sangwoo Cho, Kushal Chawla, Pengshan Cai, Zefang Liu, Chenyang Zhu, Shi-Xiong Zhang, Sambit Sahu
arXiv:2510.12306v4 Announce Type: replace
Abstract: As natural language corpora expand at an unprecedented rate, manual annotation remains a significant methodological bottleneck in corpus linguistic...
By Cameron Morin, Matti Marttinen Larsson