PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Experts

PixelMoE introduces a pixel-space generative framework that applies LLM-style Mixture-of-Experts (MoE) to diffusion models, enabling sparse scaling of model capacity without diffusion-specific routing. The approach combines Token-Choice routing, fine-grained routed experts, a shared expert, and bias-based load balancing, while also employing Pixel-Space Hierarchical Flow Matching to adjust spatial resolution during denoising. PixelMoE outperforms dense counterparts at comparable active parameter counts, achieving strong FID scores on class-conditional ImageNet-1K.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

• llms
• diffusion
• nlp
• safety