Specialization, Diversity, and Scaling in Child-Scale Language Models

The study trains small GPT‑2 models on child‑scale language data from dense home recordings (BabyView) and synthetic child‑directed dialogue (TinyDialogues), varying data amount, source, and diversity. Models trained on a single child’s environment predict that child’s language far better than others, a pattern largely driven by lexical and distributional factors as shown by a 5‑gram baseline. While training on multiple children improves predictions on pooled held‑out language, it does not yield a clear advantage on grammatical benchmarks, and naturalistic versus synthetic inputs produce distinct grammatical and lexical‑semantic patterns.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• benchmarks