Language, Language Models, and What We're Talking About

Language Models for Portuguese: A Systematic Mapping Study

The paper surveys language models created for Portuguese, noting that while rapid progress has been made in NLP, development has been uneven across languages. It systematically maps 46 Portuguese models, detailing aspects such as base model, architecture, resources, datasets, licensing, code, data, and weights. The study also traces model evolution phylogenetically, highlights research gaps, and outlines future directions for Portuguese language modeling.

Understanding the capabilities, limitations, and societal impact of large language models

Dialects of Translationese Shape Language Model Learning

The paper investigates how machine‑translated English data from 24 diverse source languages influences small English language models. It finds that source language affects model behavior: lexical diversity drives overall perplexity, while grammatical performance correlates with typological similarity to English when sufficient data is used. Additionally, translation quality strongly predicts language‑modeling performance.

Beyond the Rosetta Stone: Unification Forces in Generalization Dynamics

The paper investigates why large language models sometimes hallucinate when asked about facts in a language different from the one in which the facts were learned. By training small Transformer models on synthetic multilingual datasets, the authors show that the degree of correlation between facts and their learning language (informativeness) and the ease of language identification (extractability) determine whether models develop unified or separate representations across languages. Unified representations enable cross‑lingual fact transfer, while separate representations do not. The study proposes a unifying perspective on cross‑lingual transfer and suggests training methods to promote representational unification.

The Reformer - Pushing the limits of language modeling

The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models

arXiv:2601.19926v3 Announce Type: replace-cross Abstract: We present a systematic review of 337 articles evaluating the syntactic abilities of Transformer-based language models (TLMs), reporting on o...

From Pixels to Prompts: Vision-Language Models

arXiv:2605. 07544v3 Announce Type: replace Abstract: When you read a paper about a new Vision-Language Model today, it can be easy to forget how strange this idea would have sounded not so long ago.

GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactions

arXiv:2609.01491v1 Announce Type: cross Abstract: The growing rate at which LLM agents interact with one another raises key questions about language evolution in multi-LLM-agent settings, with implic...

The Curse of Multilinguality in Lexical Normalization

The paper investigates how many languages should be jointly trained in a single lexical normalization model. Using a fixed-capacity character-level model across twelve languages, it finds that accuracy peaks when a language is trained with only a few others—typically one to four—and then declines sharply as more languages are added, dropping about forty percent. A control experiment keeping total training data constant shows the decline is due to competition for model capacity rather than data scarcity, and no reliable typological rule predicts the optimal number of co‑training languages.

Very Large Language Models and How to Evaluate Them

surprisal is Not a Theory

The article argues that Surprisal Theory, often presented as a computational-level explanation, is not a theory in its own right. It contends that using large language model (LLM) surprisals without considering the underlying representational and algorithmic choices obscures the theory’s commitments. The authors demonstrate through three analyses that algorithm and architecture significantly influence language model probabilities, urging researchers to reassess treating LLM surprisals as interchangeable.

Causal Interventions Reveal Typologically Organized Syntactic Mechanisms in Multilingual Language Models

arXiv:2608.28924v1 Announce Type: new Abstract: Linguistic theory has long recognized cross-linguistic syntactic regularities, leading to claims that these similar structures are processed by similar...