Training a language model with 🤗 Transformers using TensorFlow and TPUs

How to train a new language model from scratch using Transformers and Tokenizers

Train and Fine-Tune Sentence Transformers Models

Introduction to Transformers: an NLP Perspective

arXiv:2311. 17633v2 Announce Type: replace-cross Abstract: Transformers have dominated empirical machine learning models of natural language processing.

How to train a Language Model with Megatron-LM

The State-Prediction Separation Hypothesis

arXiv:2607. 01218v1 Announce Type: cross Abstract: Transformers use the same forward computation stream to both predict the next token and store useful state for future token predictions.

Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression

arXiv:2606. 03465v1 Announce Type: cross Abstract: Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints.

Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression

Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, offering compact parameterizations well suited to Transformer weight structures.

Timm ❤️ Transformers: Use any timm model with transformers

Training and Finetuning Embedding Models with Sentence Transformers

Efficient training of language models to fill in the middle

How to generate text: using different decoding methods for language generation with Transformers

Fine-Tune Whisper For Multilingual ASR with 🤗 Transformers