arXiv Machine Learning

Lost but not erased: Finding traces of a forgotten language in neural speech models

The study investigates whether phonological traces of a first language persist in neural speech models after switching to a second language, mirroring phenomena observed in international adoptees. Using automatic speech recognition models trained on one language and then abruptly switched to another, researchers found that traces of the first language remained in the lowest, pre‑phonemic layers throughout second‑language training. These traces proved functional, as models with early exposure re‑learned their lost first language 14% faster than naive models, an advantage that vanished when the earliest layers were replaced with those from a non‑adopted model.

Hugging Face Trending Papers
Jun 23

Can Scale Save Us From Plasticity Loss in Large Language Models?

The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural networks capable of continual learning. Although this phenomenon has been known for decades, it has mostly been studied in older, relatively small architectures and rarely in natural-language domains.

arXiv AI
Aug 7

Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors

arXiv:2608. 06300v1 Announce Type: new Abstract: Automatic speaking assessment systems are increasingly deployed in high-stakes settings to mark second language (L2) learners' speaking tests, making it critical to show that their scores depend on speaking proficiency rather than irrelevant speaker attributes such as first language (L1) or age.

By Arya Labroo, Mengjie Qian, Kate Knill
arXiv AI
Jun 24

Can Scale Save Us From Plasticity Loss in Large Language Models?

arXiv:2606. 24752v1 Announce Type: new Abstract: The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural networks capable of continual learning.

By J. Fernando Hernandez-Garcia, Tom\'as Figliolia, Beren Millidge
arXiv Computation and Language
Sep 3

Disentangling Statistical Preemption from Entrenchment in Language Models' Avoidance of Overgeneralization

The paper investigates how language models avoid overgeneralizations by distinguishing between two types of indirect negative evidence: preemption and entrenchment. Through controlled rearing experiments on models trained on child‑caregiver conversations, the authors find that models do not exhibit verb‑specific preemption but show weak abstract preemption. Analysis of training dynamics suggests that competing structures act as indirect positive evidence rather than negative in the verb‑specific condition.

By Yixuan Wang, Freda Shi, Kanishka Misra