arXiv:2609.22633v1 Announce Type: new Abstract: Bilingual language models (LMs) offer a controlled setting for studying how training conditions shape second-language (L2) behaviour, but prior work ty...
The study compares an English-only and a bilingual decoder-only model, each 310 M parameters, trained on eight diverse languages while controlling for English exposure, compute, and document overlap. After aligning on shared English vocabulary, the authors find that token embeddings appear similar, but the deeper hidden states used for prediction diverge across models. This hidden‑state mismatch grows through middle transformer layers and persists despite controls, indicating that contextual processing differs between the models. "whyItMatters":"The findings show that embedding alignment can conceal significant internal representation differences, which is crucial for any downstream work that assumes aligned multilingual models are interchangeable."
The paper investigates whether training small decoder-only transformers on code‑switched text can induce cross‑lingual alignment. Using two 100‑million‑word multilingual corpora—one a mix of English, Dutch, and Chinese BabyBabelLM data, and another generated by inserting word‑ and sentence‑level code‑switching via an LLM—the authors find that code‑switched training aligns representations of parallel text, especially across different scripts, and that this alignment persists when later training on monolingual documents. A curriculum that progresses from word‑level code‑switching to sentence‑level code‑switching and finally to monolingual data yields models that outperform baselines on the BabyLM evaluation suite, demonstrating that code‑switching curriculum learning is an effective data augmentation strategy for multilingual pretraining.
arXiv:2509. 15001v3 Announce Type: replace-cross Abstract: Child-centered daylong recordings are essential for studying early language development, but existing speech models trained on clean adult data perform poorly due to acoustic and linguistic differences.
arXiv:2609. 17435v1 Announce Type: new Abstract: We submit M\'eTRON-FR, a 125M GPT-2 pretrained on 92.
The study investigates whether monolingual language models, trained without joint multilingual objectives, develop cross-lingual alignment. By evaluating models such as Goldfish and independently built monolingual systems, the authors find that alignable representational geometry emerges across layers, strengthening with larger data, larger models, or closer linguistic proximity. A single Procrustes rotation on parallel sentences can map hidden states between models, and applying this rotation to a German model’s residuals swaps factual predictions to those of the donor English model, demonstrating functional transfer.
The study introduces a Difference in Surprisal method that uses GPT‑2 token surprisal to automatically label telicity in English CHILDES corpora, validated against expert judgments. Logistic regression classifiers trained on 12 syntactic and lexical semantic features reveal that child speech achieves near‑perfect telicity classification using a single deterministic cue—the presence of a post‑verbal determiner—whereas adult speech relies more on verb class and other lexical semantic features, with the determiner cue neutralized. This developmental trajectory supports syntactic bootstrapping, showing that learners initially exploit high‑frequency structural cues before developing fully compositional, verb‑based event structures.
The paper reviews how large language models (LLMs) are employed to study infant syntax acquisition, focusing on the BabyLM challenge that aims for human‑level syntactic performance using developmentally realistic corpora. It critically examines dataset construction, model selection, training procedures, and syntactic evaluation methods, highlighting methodological assumptions that limit the theoretical reach of these studies. The authors find that using developmentally realistic corpora has only modest impact on benchmark performance, pointing to fundamental computational differences between LLMs and actual infant syntax learners.
We introduce Dango, a 1. 8B-parameter large language model designed for controlled studies of L1-to-L2 (Japanese-to-English) transfer in second language acquisition (SLA).
arXiv:2607. 07669v1 Announce Type: cross Abstract: Large language models increasingly \emph{understand} dialectal English, yet still \emph{produce} only standard, US-leaning English, leaving dialectal generation, the harder half of the problem, largely unaddressed.
arXiv:2609.10434v1 Announce Type: new Abstract: Self-supervised speech foundation models are now used in a wide array of downstream applications, including traditional speech recognition and as the b...
arXiv:2607. 04814v1 Announce Type: cross Abstract: Extending automatic speech recognition (ASR) to low-resource African languages is constrained by the prohibitive demands of data collection at scale.