arXiv Computer Vision

PixelDiT2: Representation-Grounded Pixel Diffusion Transformers

arXiv AI
Oct 19

Rethinking the design space of Vanilla Pixel Diffusion Transformers

The paper revisits pixel-space diffusion models, showing that a vanilla diffusion transformer can generate directly in pixel space by swapping velocity prediction for clean image prediction. It introduces MFlow, an adaptive parameterization that interpolates between these two prediction modes based on data representation and model capacity, and demonstrates competitive performance on ImageNet, UCF‑101, and text‑to‑image tasks. Additionally, the authors add ConvRefiner, a lightweight convolutional module trained post‑hoc to restore high‑frequency details while preserving FID.

By Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava, Aniruddha Mahapatra
arXiv AI
6d ago

Just Image Transformers with Late-Stage Pixel Refinement

The paper revisits JiT, a latent‑free diffusion transformer that generates images by predicting the final image. By adding simple convolutional pixel‑pathway layers after training, the authors significantly improve generation quality, showing that these layers can be activated only in the last 20% of the denoising process. On ImageNet@256, the method achieves an FID of 1.78, better than JiT’s 2.34, without extra losses.

By Denis Lukovnikov, Andreas M\"uller, Asja Fischer
arXiv AI
Oct 19

RaPD: Resolution-Agnostic Pixel Diffusion via Semantics-Enriched Implicit Representations

RaPD (Resolution‑agnostic Pixel Diffusion) introduces a continuous Neural Image Field (NIF) latent space for diffusion, enabling resolution‑free generation. It incorporates Semantic Representation Guidance to make latent learning generation‑aware and a Coordinate‑Queried Attention Renderer for scale‑aware, coordinate‑conditioned rendering. A single denoised latent can be rendered at any resolution by adjusting query coordinates, keeping diffusion cost constant while improving generation quality and scalability.

By Yanhao Ge, Shanyan Guan, Weihao Wang, Ying Tai, Mingyu You
arXiv Computer Vision
Oct 12

Learning Representations to Guide Pixel Diffusion Transformers

The paper investigates how visual representations contribute to pixel diffusion transformers and finds that higher representation quality does not always improve generative performance, and that separating architecture into specialized decoders mainly redistributes modeling rather than enhancing generative suitability. Based on these findings, the authors propose Representation Guidance (RG), a method that compacts generation-relevant information into a low-dimensional bottleneck and explicitly uses it to steer decoder generation. RG achieves state‑of‑the‑art results on ImageNet 256×256 with fewer parameters, scales efficiently, and extends to text‑to‑image tasks with strong benchmark performance.

By Zihao Pan, Mengting Chen, Cheng Shi, Taihang Hu, Zhengrong Yue, Zikang Wang, Yuan Wang, Wei Li, Sibei Yang
arXiv Computer Vision
Oct 21

PixelSR: Pixel-Space Orthogonal Flow Decoupling for Real-World Image Super-Resolution

PixelSR introduces a single‑stage Pixel Diffusion Transformer that operates directly in pixel space for real‑world image super‑resolution. It combines Pixel‑Space Conditional Diffusion—integrating low‑resolution conditioning, semantic guidance, and texture refinement—to avoid VAE compression, with Orthogonal Flow Decoupling that separates a low‑rank semantic component from a noise‑free texture component for specialized optimization. The approach yields perceptually realistic details while preserving structural fidelity, as shown by competitive results on multiple benchmarks.

By Xiang Lv, Kai Zhao, Qiang Wang, Qiao Mo, Chao Zhou, Mingwen Shao
arXiv Computer Vision
Oct 28

SpeeDiff: Scalable Pixel-Anchored End-to-End Latent Diffusion Model

SpeeDiff is a latent diffusion model that jointly trains a VAE and diffusion network from scratch, using a Tweedie Pixel Reconstruction loss to prevent latent collapse. The approach allows scaling a transformer‑based architecture and improves representation alignment. Experiments show the SpeeDiff‑XL model trains 140× faster than Vanilla SiT and 61× faster than REPA, achieving an FID of 1.50 on ImageNet 256×256 and 1.53 on 512×512 without guidance.

By Bingliang Zhang, Wenda Chu, Yizhuo Li, Linjie Yang, Yisong Yue, Katherine L. Bouman, Yang Song, Qiushan Guo