Training a language model with 🤗 Transformers using TensorFlow and TPUs

How to train a new language model from scratch using Transformers and Tokenizers

Train and Fine-Tune Sentence Transformers Models

Introduction to Transformers: an NLP Perspective

arXiv:2311. 17633v2 Announce Type: replace-cross Abstract: Transformers have dominated empirical machine learning models of natural language processing.

How to train a Language Model with Megatron-LM

The State-Prediction Separation Hypothesis

arXiv:2607. 01218v1 Announce Type: cross Abstract: Transformers use the same forward computation stream to both predict the next token and store useful state for future token predictions.

Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression

arXiv:2606. 03465v1 Announce Type: cross Abstract: Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints.