Why Transformers Need Positional Encoding For Time Series: A Visual Guide

Before Q, K, and V: Reconstructing the Transformer

Many Transformer explainers start with the finished architecture. We ask why it looks the way it does.

Functional Equivalence in Attention: A Comprehensive Study with Applications to Linear Mode Connectivity

arXiv:2606. 17830v1 Announce Type: cross Abstract: Neural network parameter spaces are inherently non-injective, as distinct parameter configurations can realize identical functions through functional equivalence.

Where does Absolute Position come from in decoder-only Transformers?

arXiv:2606. 06160v1 Announce Type: new Abstract: RoPE-trained transformers distinguish absolute position in their attention patterns, even though RoPE encodes only relative offsets in the inner product.

Pattern Formation in Transformers

arXiv:2609.37921v1 Announce Type: new Abstract: What are the inductive biases of a Transformer architecture? Existing theory on how the forward pass shapes representations either considers whether Tr...

Yes, Transformers are Effective for Time Series Forecasting (+ Autoformer)

Time-Series LLMs, Explained with t0-alpha

t0-alpha is a decoder-style patch transformer for probabilistic time-series forecasting. Raw series are split into 32-step patches, embedded, processed through causal time-attention and group-attention layers, and decoded into future quantiles rather than a single point forecast.