← Back to all news
arXiv Computation and Language By Yaning Jia, Chunhui Zhang, Chiyu Ma, Zhongyu Ouyang, Ivory Yang, Soroush Vosoughi

Scaling Makes It Possible: How Large Models Master Impossible Languages

Read the original on arXiv Computation and Language →

The paper investigates whether large language models can master "impossible languages"—languages that violate typical structural, cognitive, or logical constraints. By fine‑tuning billion‑parameter models on such languages, the authors find that large models perform comparably on impossible and possible languages, whereas smaller million‑parameter models degrade significantly, especially as semantic complexity rises. The study introduces semantic coherence as a metric and shows that scaling shifts models’ focus from grammatical rules to semantic understanding, reducing sensitivity to low‑level perturbations while preserving high‑level semantic performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Sep 10

Limits of Reliability and Scaling in Language Models

The paper argues that large language models cannot achieve perfect reliability for any task, even with unlimited scale. It establishes that each generative task has an inherent reliability ceiling set by how much output uncertainty can be resolved from observable context, with a resolvable part that can be improved by more context and a subjective part tied to task ambiguity. The authors derive a scaling law showing that performance is limited by the scarcer resource—either training data or model capacity—and explain how this law explains phenomena such as retrieval augmentation and catastrophic forgetting.

By Subhabrata Majumdar
llms
More like this →
arXiv AI
Aug 11

Scaling Inherently Interpretable Language Models

arXiv:2608. 07594v1 Announce Type: cross Abstract: Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish.

By Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail, Giang Nguyen, Isaac Plant, Muawiz Chaudhary, Nathaniel Monson, Saqib Azim, Zhichen Guo, Julius Adebayo
llmsdiffusionsafety
More like this →