Residual-Stream Burden Shapes Representation Learning in Diffusion Transformers

The paper investigates why Diffusion Transformers perform well when predicting clean data but poorly when predicting noise or velocity. It introduces the concept of *residual-stream burden*, explaining that noisy targets force the residual stream to carry noise-dependent variation through the network, limiting representation flexibility. Experiments show that spectral concentration of clean targets reduces this burden, and the authors propose Spatially Indexed Hyper-Connections (SiHC) to expand residual bandwidth, achieving state‑of‑the‑art FID on ImageNet.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms
• diffusion