The paper surveys language models created for Portuguese, noting that while rapid progress has been made in NLP, development has been uneven across languages. It systematically maps 46 Portuguese models, detailing aspects such as base model, architecture, resources, datasets, licensing, code, data, and weights. The study also traces model evolution phylogenetically, highlights research gaps, and outlines future directions for Portuguese language modeling.
By Jhessica Silva, Carlos Caetano, Helena Maia, Breno Bernard Nicolau de Fran\c{c}a, Sandra Avila, Helio Pedrini
The paper investigates how machine‑translated English data from 24 diverse source languages influences small English language models. It finds that source language affects model behavior: lexical diversity drives overall perplexity, while grammatical performance correlates with typological similarity to English when sufficient data is used. Additionally, translation quality strongly predicts language‑modeling performance.
By Jenny Kunz
The paper investigates why large language models sometimes hallucinate when asked about facts in a language different from the one in which the facts were learned. By training small Transformer models on synthetic multilingual datasets, the authors show that the degree of correlation between facts and their learning language (informativeness) and the ease of language identification (extractability) determine whether models develop unified or separate representations across languages. Unified representations enable cross‑lingual fact transfer, while separate representations do not. The study proposes a unifying perspective on cross‑lingual transfer and suggests training methods to promote representational unification.
By Carter Blum, Katja Filippova, Ann Yuan, Asma Ghandeharioun, Julian Zimmert, Fred Zhang, Jessica Hoffmann, Tal Linzen, Martin Wattenberg, Lucas Dixon, Mor Geva
arXiv:2601.19926v3 Announce Type: replace-cross
Abstract: We present a systematic review of 337 articles evaluating the syntactic abilities of Transformer-based language models (TLMs), reporting on o...
By Nora Graichen, Iria de-Dios-Flores, Gemma Boleda