Train your first Decision Transformer

Mixture of Experts (MoEs) in Transformers

Symmetry-Aware Transformer Training for Automated Planning

arXiv:2508. 07743v2 Announce Type: replace Abstract: While transformers excel in many settings, their application in the field of automated planning is limited.

Introducing Optimum: The Optimization Toolkit for Transformers at Scale

Differential Transformer V2

Vision Transformer Finetuning Benefits from Non-Smooth Components

arXiv:2602. 06883v3 Announce Type: replace Abstract: The smoothness of the transformer architecture has been extensively studied in the context of generalization, training stability, and adversarial robustness.

DiScoFormer: One transformer for density and score, across distributions