arXiv Computation and Language

Not Just an Evaluation: A Case Study of Discourse Particle Sensitivity in Language Models

The paper studies how large language models (LLMs) handle the discourse particles "just" and "only" in natural conversation. Behavioral experiments show that both humans and LLMs prefer these particles and are more sensitive to "just" than "only", while instruction tuning enhances this sensitivity. Mechanistic analysis identifies specific units in the models that respond to each particle, revealing an asymmetric overlap in their internal representations.

arXiv AI
Jul 7

The Rise of Verbal Tics in Large Language Models: A Systematic Analysis Across Frontier Models

arXiv:2604. 19139v3 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) continue to evolve through alignment techniques such as Reinforcement Learning from Human Feedback (RLHF) and Constitutional AI, a growing and increasingly conspicuous phenomenon has emerged: the proliferation of verbal tics--repetitive, formulaic linguistic patterns that pervade model outputs.

By Shuai Wu, Xue Li, Yanna Feng, Yufang Li, Zhijun Wang, Ran Wang
Hugging Face Trending Papers
Jul 13

Production and Perception in LLMs: A Token Probability Approach

The asymmetry between language production and perception has been well-documented in psycholinguistics. Whether large language models (LLMs) exhibit a functionally analogous distinction remains an open question, particularly given that LLMs rely on the same underlying mechanism (next-token prediction) for both input and output processing.

arXiv Computation and Language
Oct 21

Why Do LLMs Accept Generics? Explaining Acceptability using Psycholinguistic and Model-Derived Factors

The paper investigates why large language models (LLMs) accept or reject generic statements, drawing on psycholinguistic research that identifies factors beyond mere property frequency, such as distinctiveness and characteristicness. Using twelve LLMs evaluated on a psycholinguistic dataset and real-world generics, the study finds that models weigh property commonality against distinctiveness similarly to humans, but they prioritize factual truth and largely ignore striking properties. Manipulating hidden states along factor directions confirms that each factor influences judgments as predicted.

By Mervat Abassy, Zhaoxin Feng, Tatsuki Kuribayashi, Kentaro Inui, Emmanuele Chersoni
arXiv Computation and Language
Oct 22

Why is AI-generated language so hard to understand?

The paper investigates why AI-generated text has become harder to understand, focusing on psycholinguistic features linked to human processing difficulty such as object relative clauses, long syntactic dependencies, and low-frequency n-grams. Using a new corpus of AI-generated text from diverse domains, the study finds that frontier and near-frontier language models employ these difficult features more frequently than paraphrases from round-trip translation and human writing, including more inanimate subjects with animate verbs, longer dependencies, stacked nouns, and reduced object relatives. The authors hypothesize that AI models prioritize information density for tasks like math, code, and reasoning, diverging from human communication goals, and demonstrate this by fine‑tuning a model with a length penalty, which reduces human‑like linguistic behavior.

By Thomas Hikaru Clark, Matias B. Martin, Kyle Mahowald
arXiv AI
Jul 31

Ask don't tell: Reducing sycophancy in large language models

arXiv:2602. 23971v4 Announce Type: replace-cross Abstract: Sycophancy, the tendency of large language models to favour user-affirming responses over critical engagement, has been identified as an alignment failure, particularly in high-stakes advisory and social contexts.

By Magda Dubois, Cozmin Ududec, Christopher Summerfield, Lennart Luettgau