arXiv:2609.34240v2 Announce Type: replace-cross
Abstract: Existing open-ended generation metrics measure likelihood, lexical diversity, or distributional similarity in generic representation space, y...
By Jinnuo Liu, Junhao Zhu, Weifeng Jiang, Haoming Liu, Hongyi Wen
The study examines whether Dutch language models exhibit coherence‑illusion effects similar to human readers, using texts that refer back to earlier context with words like ‘again’ and ‘too’. Surprisal at the critical word aligns with human acceptability and eye‑tracking data, showing that models are more surprised by incoherent continuations unless a matching distractor is present. Attention entropy and an energy metric from associative‑memory literature reveal heads that behave differently under coherence versus incoherence, and ablating these heads demonstrates transfer effects across experiments, indicating a shared underlying mechanism.
By Ece Takmaz, Nitin Kumar, Li Kloostra, Jakub Dotlacil
The study evaluates eleven autoregressive transformer models on English agreement attraction scenarios using a surprisal-based approach. Results show that while transformers match human reading times for prepositional phrase configurations, they perform poorly on object‑extracted relative clauses, with predictions diverging across models and failing to capture human interference patterns. The authors argue that current transformers cannot adequately model human morphosyntactic processing and call for more rigorous, comprehensive testing to avoid misleading conclusions from limited syntactic setups.
By Titus von der Malsburg, Sebastian Pad\'o
The paper investigates how adjectival modifiers affect the semantic plausibility of events, using the Adept benchmark of 16,000 English sentence pairs that differ by a single adjective. Experiments show that sentence transformers, despite being conceptually suited to the task, underperform compared to models like RoBERTa. The authors provide an error analysis and discuss the implications of their findings for future work on balancing training and test data.
By Anna Golub, Beate Zywietz, Annerose Eichel
arXiv:2604. 25860v2 Announce Type: replace-cross Abstract: Machine-generated text (MGT) detection requires identifying structurally invariant signals across generation models, rather than relying on model-specific fingerprints.
By Lucio La Cava, Andrea Tagarelli
The paper introduces a taxonomy for the term "AI slop," a label for low‑quality AI‑generated text, and proposes interpretable dimensions to assess it. Using expert interviews and span‑level annotation, the authors find that binary slop judgments are somewhat subjective yet correlate with latent factors like coherence and relevance. Their framework can be applied to both detection and preference tasks, offering insights into linguistic and stylistic contributors to quality, while noting that fully automated, scalable methods remain an open challenge.
By Chantal Shaib, Tuhin Chakrabarty, Diego Garcia-Olano, Byron C. Wallace