arXiv AI By Subhabrata Majumdar

Information-Theoretic Limits of Reliability and Scaling in Language Models

Read the original on arXiv AI →

arXiv:2607. 14112v1 Announce Type: cross Abstract: Large language models (LLMs) are evaluated as though perfect reliability is achievable for any task given sufficient scale.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI
Jun 24

Can Scale Save Us From Plasticity Loss in Large Language Models?

arXiv:2606. 24752v1 Announce Type: new Abstract: The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural networks capable of continual learning.

By J. Fernando Hernandez-Garcia, Tom\'as Figliolia, Beren Millidge
Hugging Face Trending Papers
Jun 23

Can Scale Save Us From Plasticity Loss in Large Language Models?

The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural networks capable of continual learning. Although this phenomenon has been known for decades, it has mostly been studied in older, relatively small architectures and rarely in natural-language domains.