← Back to all news
arXiv Machine Learning By Yingheng Wang, Yaoqiang Li, Yaqin Wu, Junwen Bai, Jiatao Gu, Christopher De Sa, Volodymyr Kuleshov

Purify Before You Align: Improving Representation Alignment for Diffusion Models

Read the original on arXiv Machine Learning →

The paper proposes Local Appearance Purification (LAP), a method that refines the alignment target for diffusion models by removing locally predictable content and focusing on the residual. LAP improves generation quality across multiple pretrained encoders, notably reducing FID from 53.1 to 42.3 on ImageNet-256 with MAE. The authors provide a theoretical framework linking alignment effectiveness to the information emphasized by the target and introduce a constant‑readout test to predict purification benefits.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 8

ORCA: Hunting Compositional Failures in Text-to-Image Diffusion

ORCA (Orthogonal Residual Compositional Alignment) is a method that improves text-to-image diffusion models by aligning the diffusion transformer’s latent space with a low‑rank target derived from a frozen visual encoder. It introduces an auxiliary loss that uses a predictor with an orthogonal basis parameterised by a learned residual between T5 and CLIP embeddings, providing a prompt‑dependent signal for selecting the visual readout subspace. Experiments on three diffusion‑transformer backbones show that ORCA improves FID and GenEval scores, especially on attribute binding, spatial relations, and multi‑object prompts, without adding inference‑time cost.

By Arshia Hemmat, Amirhossein Vahidi, Amitis Shidani, Mohammad Vali Sanian, Hesam Asadollahzadeh, Aryan Yazdan Parast, Mohammad Lotfollahi
llmsragdiffusionmultimodalsafety
More like this →