arXiv Machine Learning By Zetai Cen, Jin Zhu, Xinwei Shen, Chengchun Shi

Perturbation is All You Need for Extrapolating Language Models

Read the original on arXiv Machine Learning →

arXiv:2605. 04344v2 Announce Type: replace-cross Abstract: This paper develops a statistical theory of extrapolation for large language models, by reinterpreting them through pre-post-additive noise models.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI
Jun 16

Data Augmentations for Data-Constrained Language Model Pretraining

arXiv:2606. 16246v1 Announce Type: cross Abstract: As AI labs approach a data ceiling where compute capacity outpaces the rate of new high-quality text generation, language model pretraining is shifting toward a data-constrained, compute-abundant regime that demands productive multi-epoch training on fixed corpora.

By Michael K. Chen, Xikun Zhang, Zhen Wang
arXiv Machine Learning
1d ago

Language models suffer from a curse of ambiguity

arXiv:2608. 15448v1 Announce Type: cross Abstract: Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever.

By Nicolas Zucchet, Hyun Dong Lee, Scott Linderman
arXiv Machine Learning
22h ago

Large Language Models: A Mathematical Formulation

arXiv:2601. 22170v2 Announce Type: replace-cross Abstract: Large language models (LLMs) process and predict sequences containing text to answer questions, and address tasks including document summarization, providing recommendations, writing software and solving quantitative problems.

By Ricardo Baptista, Andrew Stuart, Son Tran