Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion
Read the original on arXiv Machine Learning →The paper presents a theoretical analysis of score mismatch in discrete diffusion models, deriving a KL divergence upper bound that quantifies the bias introduced when the training and target distributions differ. It examines partial-observation constraints such as text infilling and sentence completion, showing that the Conditional SEDD sampler can incur a bias that grows with dimension, vocabulary size, and compatibility. To mitigate this, the authors propose a Randomized Hard Clamping (RHC) sampler for coordinatewise set constraints, demonstrating through experiments that it effectively reduces bias and improves sampling performance.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.