The paper investigates how adjectival modifiers affect the semantic plausibility of events, using the Adept benchmark of 16,000 English sentence pairs that differ by a single adjective. Experiments show that sentence transformers, despite being conceptually suited to the task, underperform compared to models like RoBERTa. The authors provide an error analysis and discuss the implications of their findings for future work on balancing training and test data.
By Anna Golub, Beate Zywietz, Annerose Eichel
arXiv:2606.11350v2 Announce Type: replace
Abstract: Retrieval-augmented generation (RAG) degrades when applied to large, heterogeneous document collections. In RAG systems, dense similarity becomes l...
By Nabaraj Subedi, Ahmed Abdelaty, Shivanand Venkanna Sheshappanavar
arXiv:2512. 04144v2 Announce Type: replace Abstract: Targeted interventions on language models, such as unlearning or model editing, aim to modify specific information, but their effects often propagate to related, unintended areas (e.
By Roy Rinberg, Usha Bhalla, Igor Shilov, Flavio P. Calmon, Rohit Gandikota
The paper introduces a dual‑dimensional framework called Automated Item Similarity Analysis (AISA) that uses Large Language Models to assess incidental content similarity in large‑scale assessments. It combines Structured Decomposition and Semantic Relatedness to capture both structural and semantic nuances that traditional metrics miss. Psychometric validation shows that LLM‑derived metrics better align with construct‑irrelevant local dependence and produce more coherent item groupings, and simulations in Computerized Adaptive Testing demonstrate improved estimation stability and reduced bias with minimal efficiency loss.
By Jing Huang, Jihong Zhang, Hua-Hua Chang
arXiv:2608. 10216v1 Announce Type: cross Abstract: Agent frameworks ship quality gates that compare text blocks by embedding-cosine similarity and decide at a fixed cutoff.
By Scott E. Frias
MUDDLE is a benchmark designed to disentangle the effects of document length and topical distractors on document question‑answering systems. It contains 270 human‑annotated questions, each tested in five conditions: the source alone, the source with two or four hard negatives (topically similar), and the source with two or four random distractors matched in length and provenance. Experiments with GPT‑5‑mini show that hard negatives reduce accuracy more than length‑matched random distractors, indicating that topical similarity is a more significant source of error than length alone.
By Jason Luo, Saibilila Abudukelimu, Judy Song, Andrew Feng, Shivank Garg, Vasu Sharma, Kevin Zhu
arXiv:2608. 10101v1 Announce Type: cross Abstract: Code review is credited with substantially changing a patch's code between its first submission and the version that eventually lands.
By Lucas Ciziks, Paulo Meirelles, Marco Aur\'elio Gerosa
The paper investigates how semantic meaning is arranged locally within large language models. By using two controlled probes—one cross‑lingual and one refinement probe—on four decoder‑only LLMs, it finds that after removing dominant variance directions, sentence pairs rank by semantic relatedness, semantic factors cluster around an invariant core with structured variation, refinement tightens neighborhoods, and sentence‑level versus token‑local readouts reveal complementary geometric patterns. These results suggest that LLM semantics form local anisotropic neighborhoods that contract when meaning is refined.
By Andr\'es Daniel Godoy Ortiz
arXiv:2609.08609v2 Announce Type: replace
Abstract: Tracking semantic change in low-resource languages across extensive historical timelines presents significant challenges due to data scarcity and t...
By Nevidu Jayatilleke, Nisansa de Silva
The paper introduces VOLM, a framework that quantifies how much original value a human adds to a document beyond what a language model could generate from a task description alone. Unlike existing tools that focus on stylistic detection, VOLM extracts content at varying granularities, reconstructs it with an LLM, and compares these reconstructions to those derived from the task description. Evaluations across news articles, ICLR peer reviews, and argumentative essays show that VOLM can distinguish human-authored texts from LLM-generated ones while remaining robust to content-preserving transformations.
By Vibhhu Sharma, Thorsten Joachims, Sarah Dean
arXiv:2608.21702v1 Announce Type: new
Abstract: Retrieval-Augmented Generation (RAG) grounds LLM generation on retrieved documents, but the standard terminal retrieval stage--dense-vector similarity,...
By Jing Liu, Yongxing Qi, Muchen Jiang, Chengnan Hu, Qingqing Peng, Haoming Wang, Yuqing Wang, Yang Yu, Xu Zhang, Ting Wu
arXiv:2605.29001v2 Announce Type: replace-cross
Abstract: A paraphrase-quality audit of MathCheck (ICLR 2025) detected 4 semantically incorrect paraphrases in 129 groups (3.1%); removing them drops G...
By Nishal Thomas, Noel Thomas