The paper examines how language models learn action nominal constructions (ANCs) across 96 synthetic target languages with varied grammatical features. Using GPT‑2‑small trained from scratch, the study evaluates models on minimal pairs covering 34 grammatical phenomena, revealing that models mirror typological patterns in basic clause configurations but diverge in complement systems and ANC strategies. The findings suggest that natural language distributions may arise from both general learning pressures and functional demands not fully captured by current model experiments.
By Iskar Deng, Shane Steinert-Threlkeld
arXiv:2610.33433v1 Announce Type: new
Abstract: Linguistic productivity, as evidenced by the use of words in novel contexts, is one of the most striking facts about human language. However, how abstr...
By Michael Wilson, Robert Frank
The study examines Urdu light verbs, which add schematic event meaning while staying lexically linked to their main verbs. Using contextual embeddings from UrduBERT, DunbaaBERT, and multilingual BERT on 1,126 sentences, the authors find significant representational separation between main and light uses across all verb–model pairs, yet main and light uses of the same lemma remain closer than mismatched pairs. In a seven‑way prediction task limited to light uses, UrduBERT achieves 0.866 accuracy and 0.852 macro‑F1, and maintains 0.782 accuracy when tested on unseen preceding forms, demonstrating generalization beyond local verb combinations.
By Farah Adeeba, Miriam Butt
arXiv:2606. 07479v1 Announce Type: cross Abstract: Turkish idiomatic light verb constructions (LVCs) are challenging for multiword expression processing because they often share the same surface form as fully literal verb-object combinations while functioning as a single, partially idiomatic predicate.
By Sercan Karaka\c{s}, Yusuf \c{S}im\c{s}ek
The study uses sparse autoencoders (SAEs) to probe how the Pythia-70M language model encodes grammatical number. By identifying and manipulating SAE‑derived features such as "plural noun" and "generate a plural verb," the authors show that removing these features forces the model to produce singular verbs, and vice versa. Similar asymmetries were observed in larger models (Gemma‑2‑2b and Llama‑3.1‑8b), indicating a privative representation of number in English.
By Jacob Solinsky (University of Minnesota), Serguei Pakhomov (University of Minnesota)
The paper presents a framework that combines human-curated vocabularies with a few human-written exemplars to enable large language models (LLMs) to generate coherent text in extremely low-resource or unseen languages. By sampling candidate words from a Zipfian distribution and providing syntactic signals, the method achieves fluent sentence-level generation and promising paragraph-level results across languages with varying resource levels. The study evaluates output quality using lexical diversity, off-target ratio, and human fluency judgments, demonstrating that reasoning-enabled LLMs can effectively bootstrap synthetic data even when digital resources are scarce.
By Michael Christlambert Sinanta (Independent Researcher), Muhammad Dehan Al Kautsar (Mohamed bin Zayed University of Artificial Intelligence), Amanda Nurul Izzah (Independent Researcher), Thanh-Nhi Nguyen (Independent Researcher), Phan Phuong Mai Chau (Mohamed bin Zayed University of Artificial Intelligence), Zheng Wei Lim (University of Amsterdam), Fajri Koto (Mohamed bin Zayed University of Artificial Intelligence)
The study probes Gemma‑2‑9B‑IT with Sparse Autoencoders across English, Hebrew, and Russian to examine how multilingual LLMs handle informal register. By using a dataset of polysemous terms that appear in literal and informal contexts, the authors isolate pragmatic register processing from lexical cues. They discover a small, robust cross‑linguistic core that forms an informal register subspace, which becomes clearer in deeper layers and can causally shift output formality across all tested languages, even transferring zero‑shot to six unseen languages.
By Uri Z. Kialy, Avi Shtarkberg, Ayal Klein
arXiv:2610.32921v1 Announce Type: new
Abstract: Modern language models rely on subwording tokenization to define the discrete units of input processing, typically using methods like Byte Pair Encodin...
By Pietro Di Nicco, Lucio La Cava, Andrea Tagarelli
The paper introduces the Latent Core Tokenizer (LCT), a language‑agnostic method that first discovers reusable linguistic units using Minimum Description Length, entropy‑based boundary signals, and morphotactic constraints before building a shared vocabulary. With a 200K‑token vocabulary across 104 languages, LCT shows lower fertility and higher MorphScore than BPE, Unigram, and parity‑aware BPE, while keeping tokenization cost comparable across languages. On four multilingual downstream benchmarks, LCT outperforms the baselines by 1.48, 1.83, and 2.00 aggregate points, demonstrating that compression alone does not guarantee representation quality and underscoring the role of morphology‑driven structural discovery.
By Felermino D. M. A. Ali, Millicent Ochieng, Ogbemi Ekwejunor-Etchie, Ade Famoti, Jacki O'Neill, Debjit Paul
arXiv:2610.16601v1 Announce Type: cross
Abstract: Processing multilingual text requires both identifying a broad range of languages and distinguishing the language of each token in inputs containing...
By Seongtae Hong, Youngjoon Jang, Seungyoon Lee, Jungseob Lee, Heuiseok Lim
arXiv:2610.08037v1 Announce Type: new
Abstract: Language models frequently generate outputs in unintended languages or scripts, a phenomenon known as off-target generation. While existing research ha...
By David Kletz, Sandra Mitrovi\'c, Ljiljana Dolami\'c, Fabio Rinaldi
arXiv:2601.19926v3 Announce Type: replace-cross
Abstract: We present a systematic review of 337 articles evaluating the syntactic abilities of Transformer-based language models (TLMs), reporting on o...
By Nora Graichen, Iria de-Dios-Flores, Gemma Boleda