← Back to all news
arXiv Computation and Language By Victor Mazzotti, Luiz Pereira, Marina Bitencourt dos Santos, Helena Maia, Carlos Caetano, N\'adia Felix, Sandra Avila

Does Linguistic Structure Enrichment Enhance Coherence Assessment? Not With Current Architectures

Read the original on arXiv Computation and Language →

The paper examines whether adding syntactic and rhetorical structure to text can improve the prediction of incoherence in large language model outputs. Experiments show that plain text actually yields higher accuracy, as the added structural information conflicts with the models’ architectures. The authors also demonstrate that coherence assessment can help detect misleading content by applying zero‑shot experiments to a Brazilian disinformation dataset.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
4d ago

When Context Misleads: Surprisal, Energy and Attention Entropy as Metrics of Coherence Illusions in LLMs

The study examines whether Dutch language models exhibit coherence‑illusion effects similar to human readers, using texts that refer back to earlier context with words like ‘again’ and ‘too’. Surprisal at the critical word aligns with human acceptability and eye‑tracking data, showing that models are more surprised by incoherent continuations unless a matching distractor is present. Attention entropy and an energy metric from associative‑memory literature reveal heads that behave differently under coherence versus incoherence, and ablating these heads demonstrates transfer effects across experiments, indicating a shared underlying mechanism.

By Ece Takmaz, Nitin Kumar, Li Kloostra, Jakub Dotlacil
llms
More like this →
arXiv Computation and Language
Aug 31

Diverging Transformer Predictions for Human Sentence Processing: A Comprehensive Analysis of Agreement Attraction Effects

The study evaluates eleven autoregressive transformer models on English agreement attraction scenarios using a surprisal-based approach. Results show that while transformers match human reading times for prepositional phrase configurations, they perform poorly on object‑extracted relative clauses, with predictions diverging across models and failing to capture human interference patterns. The authors argue that current transformers cannot adequately model human morphosyntactic processing and call for more rigorous, comprehensive testing to avoid misleading conclusions from limited syntactic setups.

By Titus von der Malsburg, Sebastian Pad\'o
llmsbenchmarks
More like this →
arXiv AI
Sep 1

Using Prosody to Predict Syntactic Structure

arXiv:2608.30260v1 Announce Type: cross Abstract: While it is well-established that prosody carries crucial cues for syntactic structure, the degree and nature of correspondence between these two dom...

By Junghyun Min, Alex Warstadt, Tamar I. Regev, Tiago Pimentel, Ethan Gotlieb Wilcox
llmsmultimodal
More like this →