arXiv Computation and Language August 31, 2026 By Emily Cheng, Ryan Cotterell

A Formal Limitation on Learning Human Language From Textual Corpora

Read the original on arXiv Computation and Language →

The paper investigates whether a listener can infer a speaker’s intended meaning solely from the form of an utterance, using an information‑theoretic framework. By modeling language as a joint distribution over meanings, contexts, and utterances, the authors derive upper bounds on the probability that any text‑based featurizer—such as the hidden states of large language models—can recover the intended meaning. These bounds depend on the irreducible uncertainty left by form and the portion that can only be resolved by extralinguistic context, implying that no representation, regardless of data or supervision, can exceed these limits. Experiments on artificial languages, Mandarin zero‑pronoun resolution, and color reference support the theoretical predictions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

llms

arXiv Machine Learning

Jul 7

The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices

arXiv:2603. 18482v2 Announce Type: replace-cross Abstract: Standard decoding strategies for text generation, including top-$k$, nucleus sampling, and contrastive search, select tokens based on likelihood, restricting outputs to high-probability regions.

By Esteban Garces Arias, Nurzhan Sapargali, Christian Heumann, Matthias A{\ss}enmacher

llms safety

Hugging Face Trending Papers

Aug 18

Language Has Two Parameters: Narrative-Induced Semantic Plasticity and Phase-Sensitive Interpretation

The paper argues that language operates with two parameters: amplitude, which measures how often words co‑occur, and phase, a signed relational factor that determines how co‑activated meanings combine and can reverse a meaning’s contribution. Unlike amplitude, phase is not captured by standard word embeddings or transformer attention weights and is indexed to individuals and dyadic interactions. The authors propose six empirical predictions to test phase’s role and suggest that future language models should incorporate agent‑indexed, phase‑bearing semantic states.

llms rag agents safety

arXiv AI

Jul 23

Internal Knowledge Without External Expression: Probing the Generalization Boundary of a Classical Chinese Language Model

arXiv:2604. 14180v2 Announce Type: replace-cross Abstract: We train a 318M-parameter Transformer language model from scratch on a curated corpus of 1.

By Jiuting Chen, Yuan Lian, Hao Wu, Tianqi Huang, Hiroshi Sasaki, Makoto Kouno, Jongil Choi

llms reinforcement-learning

arXiv Computation and Language

Oct 27

Large Language Model probabilities cannot reliably distinguish between possible and impossible language

arXiv:2509.15114v2 Announce Type: replace Abstract: A controversial test for Large Language Models concerns the ability to discern possible from impossible language. While some evidence attests to th...

By Evelina Leivada, Raquel Montero, Paolo Morosi, Natalia Moskvina, Tamara Serrano, Nikoleta Pantelidou, Marcel Aguilar, Fritz Guenther

llms benchmarks

arXiv Computation and Language

Oct 22

Why is AI-generated language so hard to understand?

The paper investigates why AI-generated text has become harder to understand, focusing on psycholinguistic features linked to human processing difficulty such as object relative clauses, long syntactic dependencies, and low-frequency n-grams. Using a new corpus of AI-generated text from diverse domains, the study finds that frontier and near-frontier language models employ these difficult features more frequently than paraphrases from round-trip translation and human writing, including more inanimate subjects with animate verbs, longer dependencies, stacked nouns, and reduced object relatives. The authors hypothesize that AI models prioritize information density for tasks like math, code, and reasoning, diverging from human communication goals, and demonstrate this by fine‑tuning a model with a length penalty, which reduces human‑like linguistic behavior.

By Thomas Hikaru Clark, Matias B. Martin, Kyle Mahowald

llms fine-tuning

Hugging Face Trending Papers

Oct 14

Articulation Upper Bounds for Language Models

Language captures only a subset of the perceptions, intuitions and intents that form meaning between speakers. What, then, are the limits of language? We take steps towards measuring this in LLMs as f...