PixelDiT2: Representation-Grounded Pixel Diffusion Transformers
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The paper revisits pixel-space diffusion models, showing that a vanilla diffusion transformer can generate directly in pixel space by swapping velocity prediction for clean image prediction. It introduces MFlow, an adaptive parameterization that interpolates between these two prediction modes based on data representation and model capacity, and demonstrates competitive performance on ImageNet, UCF‑101, and text‑to‑image tasks. Additionally, the authors add ConvRefiner, a lightweight convolutional module trained post‑hoc to restore high‑frequency details while preserving FID.
The paper revisits JiT, a latent‑free diffusion transformer that generates images by predicting the final image. By adding simple convolutional pixel‑pathway layers after training, the authors significantly improve generation quality, showing that these layers can be activated only in the last 20% of the denoising process. On ImageNet@256, the method achieves an FID of 1.78, better than JiT’s 2.34, without extra losses.
RaPD (Resolution‑agnostic Pixel Diffusion) introduces a continuous Neural Image Field (NIF) latent space for diffusion, enabling resolution‑free generation. It incorporates Semantic Representation Guidance to make latent learning generation‑aware and a Coordinate‑Queried Attention Renderer for scale‑aware, coordinate‑conditioned rendering. A single denoised latent can be rendered at any resolution by adjusting query coordinates, keeping diffusion cost constant while improving generation quality and scalability.
arXiv:2608. 09133v1 Announce Type: cross Abstract: Image super-resolution (SR) with large generative models has recently achieved remarkable perceptual quality, yet maintaining fidelity to the LR observation remains challenging.
Unified image restoration (UIR) aims to recover high-quality (HQ) content from low-quality (LQ) images with different degradations using a single model. Most recent methods adapt large pretrained text-to-image (T2I) latent diffusion models for their strong capacity and generative priors.
The paper investigates how visual representations contribute to pixel diffusion transformers and finds that higher representation quality does not always improve generative performance, and that separating architecture into specialized decoders mainly redistributes modeling rather than enhancing generative suitability. Based on these findings, the authors propose Representation Guidance (RG), a method that compacts generation-relevant information into a low-dimensional bottleneck and explicitly uses it to steer decoder generation. RG achieves state‑of‑the‑art results on ImageNet 256×256 with fewer parameters, scales efficiently, and extends to text‑to‑image tasks with strong benchmark performance.