← Back to all news
arXiv Computer Vision By Benyuan Meng, Longtao Huang, Yang Liu, Liang Lin

We Have Been Interpreting Diffusion Transformer Backbone Behaviors Wrong

Read the original on arXiv Computer Vision →

The paper revisits the interpretation of depth profiles in Diffusion Transformer backbones, showing that the commonly observed inverted‑U curve reflects the influence of the prediction target (e.g., ε, u, x0) rather than a simple shift from coarse to fine semantic granularity. It argues that the semantically meaningful signal is merely an intermediate by‑product of predicting the target, and that different training setups—particularly the choice of prediction target—predictably alter the curve’s shape. The authors demonstrate how this new understanding can guide the optimal placement of REPA modules and provide code for further exploration.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Jun 17

Rethinking Cross-Layer Information Routing in Diffusion Transformers

arXiv:2605. 20708v2 Announce Type: replace-cross Abstract: Diffusion Transformers (DiTs) have become a de facto backbone of modern visual generation, and nearly every major axis of their design -- tokenization, attention, conditioning, objectives, and latent autoencoders -- has been extensively revisited.

By Chao Xu, Maohua Li, Qirui Li, Yixuan Xu, Yanke Zhou, Yunhe Li, Cuifeng Shen, Hanlin Tang, Kan Liu, Tao Lan, Lin Qu, Shao-Qun Zhang
llmsdiffusionnlpfine-tuningefficiencysafety
More like this →
arXiv Computation and Language
Sep 2

Measuring Optimal Transport in Transformer Depth

The study investigates whether transformer models transport token representations across layers in a manner consistent with optimal transport theory. Using Pythia-160m and Pythia-410m, the authors measure exact assignments between consecutive layer clouds, calibrate couplings, and decompose the transport cost into common shifts and token‑specific moves. Results show that at the final layer both models align closely with the optimal‑transport map—Pythia‑410m at optimal cost and Pythia‑160m slightly above—while earlier layers deviate, and the agreement improves with training.

By Alexandre Quemy
llms
More like this →
arXiv Computer Vision
Oct 12

Why DiT Models Underperform as Representation Learners without Long Skip Connections

The paper investigates why Diffusion Transformers (DiT) produce weaker features for vision tasks compared to their strong generative performance. Through controlled training, the authors demonstrate that incorporating long skip connections (LSCs) significantly improves feature quality. They explain that LSCs allow diffusion noise to bypass the main backbone, preventing noise pollution of image signals and aligning the architecture more closely with the diffusion objective.

By Benyuan Meng, Yiliang Zhang, Jin-Wen Wu, Qianqian Xu, Qingming Huang, Longtao Huang
llmsdiffusion
More like this →
arXiv Machine Learning
Oct 8

Residual-Stream Burden Shapes Representation Learning in Diffusion Transformers

The paper investigates why Diffusion Transformers perform well when predicting clean data but poorly when predicting noise or velocity. It introduces the concept of *residual-stream burden*, explaining that noisy targets force the residual stream to carry noise-dependent variation through the network, limiting representation flexibility. Experiments show that spectral concentration of clean targets reduces this burden, and the authors propose Spatially Indexed Hyper-Connections (SiHC) to expand residual bandwidth, achieving state‑of‑the‑art FID on ImageNet.

By Tongtong Liang, Siqi Kou, Ziqiao Xi, Esha Singh, Kun Zhou, Zhijie Deng, Alexander Cloninger, Yu-Xiang Wang, Rahul Parhi
llmsdiffusion
More like this →