The paper investigates whether language models can transfer hierarchical syntactic knowledge from a source language (L1) to a target language (L2) using a novel cross‑lingual poverty‑of‑stimulus (CPOS) experimental design. Results show that models can partially transfer L1 structural evidence to L2, producing a small but detectable hierarchical bias that varies across language pairs. The study further demonstrates that vocabulary overlap is crucial for successful transfer, indicating that cross‑lingual learning of hierarchical generalization is conditionally achievable.
arXiv:2608.28924v1 Announce Type: new Abstract: Linguistic theory has long recognized cross-linguistic syntactic regularities, leading to claims that these similar structures are processed by similar...
The paper explores structural transfer, where models are first trained on non-language data such as music, probabilistic grammars, and cellular automata to induce priors for natural language tasks. This pretraining acts as a weight initialization for multilingual language modeling and leads to lower next-token prediction loss and smaller weight shifts during subsequent language training. However, the improved loss does not consistently translate into better downstream linguistic performance, and the efficiency of non-language data is lower than that of additional language data.
Large language models show strong cross‑lingual abilities, but previous studies have examined them only at isolated training points. This work pretrains a 1.7B multilingual model on nine languages, recording checkpoints at fine granularity to track translation development. Using word‑level translation and a new dataset, the authors find that early learning is dominated by token‑level copying and surface similarities, while later stages involve more general translation mechanisms and refined copying.
arXiv:2510. 15551v2 Announce Type: replace-cross Abstract: Any piece of knowledge is usually expressed in one or a handful of natural languages on the web or in any large corpus.
The paper investigates why large language models sometimes hallucinate when asked about facts in a language different from the one in which the facts were learned. By training small Transformer models on synthetic multilingual datasets, the authors show that the degree of correlation between facts and their learning language (informativeness) and the ease of language identification (extractability) determine whether models develop unified or separate representations across languages. Unified representations enable cross‑lingual fact transfer, while separate representations do not. The study proposes a unifying perspective on cross‑lingual transfer and suggests training methods to promote representational unification.
arXiv:2606. 18033v1 Announce Type: cross Abstract: Cross-lingual transfer in multilingual NLP has been widely explored in supervised fine-tuning contexts, where factors like data availability and linguistic similarity largely determine transfer quality.
arXiv:2601.16390v2 Announce Type: replace-cross Abstract: Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant la...
arXiv:2610.07168v1 Announce Type: new Abstract: Representations of translated sentences are similar in the inner layers of multilingual language models -- an observation connected to the platonic rep...
arXiv:2607. 22699v1 Announce Type: new Abstract: As Large Language Models (LLMs) grow more capable across diverse tasks, their (in)ability to generalize remains difficult to quantify and poorly understood beyond limited domains.
The paper investigates cross‑lingual transfer for sequential sentence classification (SSC) in research papers, focusing on 13 non‑English languages. Experiments show that linguistic proximity does not reliably predict transfer success, whereas structural similarity in rhetorical organization—particularly label distribution similarity—correlates positively with performance. The authors introduce three generative‑model methods that exploit structural cues, achieving parity with strong encoder baselines on‑domain and outperforming them when transferring to unseen languages.
arXiv:2609.01356v1 Announce Type: new Abstract: Multilingual large language models (mLLMs) achieve strong performance in machine translation, yet our understanding of the mechanisms by which they tra...
arXiv:2609.37121v1 Announce Type: new Abstract: Cross-linguistic effects are a central topic in bilingual first-language acquisition. Artificial learners can help investigate L1-L2 interactions by en...