The paper argues that the concrete random noise used in diffusion models is not merely a passive perturbation but a learnable input that can be exploited by the model. By analyzing how clean data and realized noise jointly form the noisy input, the authors show that the model can learn regularities in the data or in the noise structure, and that these two routes can interact. Experiments on MNIST and CIFAR‑10 using pseudorandom streams demonstrate that structured‑noise training can reduce prediction loss, but this advantage disappears when test noise is replaced with IID noise, indicating that the learned dependence is tied to the specific noise structure.
By Shengzhi Deng, Chenqi Ye, Yanze Guo
arXiv:2510. 17917v2 Announce Type: replace-cross Abstract: Data unlearning aims to remove the influence of specific training samples from a trained model.
By Jinseong Park, Mijung Park
arXiv:2608. 02575v1 Announce Type: new Abstract: Diffusion models rely on stochastic inputs, yet on finite-precision hardware, the "randomness" they consume is realized as deterministic numerical orbits generated by pseudorandom rules.
By Shengzhi Deng, Chenqi Ye, Yanze Guo
The paper introduces a new method for training data attribution in diffusion models called TID, which uses a local score discrepancy measure and can be estimated without retraining. It further distills this approach into TIDE, a forward‑only student that reproduces the teacher’s rankings using internal activations, achieving comparable accuracy at dramatically lower query cost. Experiments on CIFAR‑10, ArtBench‑10, and MS‑COCO show that TID outperforms existing methods and TIDE attributes samples in milliseconds, faster than generation itself.
By Shixuan Liu, Joan Serr\`a, Kin Wai Cheuk, Jinju Kim, Woosung Choi, Yukara Ikemiya, Wei-Hsiang Liao, Jiaqi W. Ma, Yuki Mitsufuji
arXiv:2606. 09718v1 Announce Type: new Abstract: Diffusion models have demonstrated remarkable generative capabilities and have also emerged as powerful self-supervised representation learners, yet the connection between these two abilities remains less explored.
By Xiao Li, Yixuan Jia, Zekai Zhang, Xiang Li, Lianghe Shi, Jinxin Zhou, Zhihui Zhu, Liyue Shen, Qing Qu
Uniform discrete diffusion models (UDMs) typically rely on explicit time conditioning, yet this study finds that such conditioning is often unnecessary in practice. While the population‑optimal UDM predictor generally depends on time—controlling how much the model should trust the observed context—the dependence becomes negligible in finite‑data language settings. Empirical results show that trained language UDMs exhibit limited time sensitivity across most of the diffusion trajectory, and time‑agnostic predictors can match or outperform time‑conditioned models on various datasets and training objectives.
By Chunsan Hong, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Jong Chul Ye, Yuki Mitsufuji
arXiv:2602. 02908v2 Announce Type: replace-cross Abstract: Diffusion models trained on different, non-overlapping subsets of a dataset often produce strikingly similar outputs when given the same noise seed.
By Binxu Wang, Jacob Zavatone-Veth, Cengiz Pehlevan
The paper investigates memorization in diffusion models, showing that the empirical score function is a weighted sum of Gaussian score functions with sharp softmax weights, causing individual training samples to dominate and lead to sampling collapse. By approximating this function with a neural network, the authors obtain a smoother representation that generalizes better. They introduce two techniques—Noise Unconditioning and Temperature Smoothing—to further reduce single‑sample dominance, and demonstrate improved generalization across multiple datasets while preserving generation quality.
By Xinyu Zhou, Jiawei Zhang, Stephen J. Wright
arXiv:2609.37974v1 Announce Type: cross
Abstract: Masked diffusion models (MDMs) generate text by unmasking several tokens per step, but they are trained and sampled under different conditions. The m...
By Manuel Madeira, Amitis Shidani, Alice Bizeul, Victor Turrisi, Louis B\'ethune, Bhavika Devnani, Dan Busbridge, Pierre Ablin, Jo\~ao Monteiro
arXiv:2605.12597v3 Announce Type: replace-cross
Abstract: Computational sampling has been central to the sciences since the mid-20th century. While machine-learning-based approaches have recently ena...
By Luca Maria Del Bono, Giulio Biroli, Patrick Charbonneau, Marylou Gabri\'e
The paper investigates when language diffusion models, specifically Uniform-based Discrete Diffusion Models (UDDMs), shift from memorizing training data to generalizing to new data. It shows that UDDMs act as associative memories, forming basins of attraction around stored examples without requiring an explicit energy function. By measuring token recovery and conditional entropy, the authors identify a sharp transition governed by training set size, where memorization (vanishing entropy) gives way to generalization (finite entropy).
By Bao Pham, Mohammed J. Zaki, Luca Ambrogioni, Dmitry Krotov, Matteo Negri
The paper examines how Denoising Diffusion Probabilistic Models (DDPMs) perform on globally constrained discrete tasks such as Sudoku and N-queens. It shows that standard diffusion sampling, which keeps updates close to the noisy state, often preserves early mistakes, whereas sampling directly from the model’s clean predictions dramatically improves validity (e.g., Sudoku from 31% to 95%). The authors further introduce self‑correction training, exposing the model to its own predictions to reduce inference errors, which enhances the performance of standard samplers across tasks.
By Mariia Drozdova, St\'ephane Liem Nguyen, Fran\c{c}ois Fleuret