Transformer-based Encoder-Decoder Models

T5Gemma: A new collection of encoder-decoder Gemma models

Introducing T5Gemma, a new collection of encoder-decoder LLMs.

On the quantitative analysis of decoder-based generative models

How to generate text: using different decoding methods for language generation with Transformers

Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models

Geometry-Preserving Encoder/Decoder in Latent Generative Models

arXiv:2501. 09876v3 Announce Type: replace-cross Abstract: Generative modeling aims to generate new data samples that resemble a given dataset.

Understanding Rate-Distortion Performance in Distributed Transformer Inference

arXiv:2601. 22002v5 Announce Type: replace Abstract: Transformers achieve superior performance on many tasks, but impose heavy compute and memory requirements during inference.

Ettin Suite: SoTA Paired Encoders and Decoders

Universal Assisted Generation: Faster Decoding with Any Assistant Model

Introducing RWKV - An RNN with the advantages of a transformer

Remote VAEs for decoding with Inference Endpoints 🤗

UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reduction

arXiv:2608. 01298v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) have emerged as a core architecture in generative modeling due to their scalability and adaptability to multimodal tasks.

Introducing Gemma 4 12B: a unified, encoder-free multimodal model