Block Sparse Matrices for Smaller and Faster Language Models

Very Large Language Models and How to Evaluate Them

Efficient training of language models to fill in the middle

Large Language Models: A New Moore's Law?

Break Through the Compression Bottleneck: From Theory to Practice

arXiv:2607. 20434v1 Announce Type: cross Abstract: As the parameter size of language models continues to grow, effective model compression is required to reduce their computational and memory overhead.

Riemannian Gradient Descent for Low-Rank Architectures

arXiv:2606. 02328v1 Announce Type: new Abstract: We explore Riemannian optimization techniques for rank-factored matrix parameters, targeting contemporary deep learning applications.

Evaluating large language models trained on code