arXiv Machine Learning

Discrete Diffusion Inference-Time Control with Nested Sequential Monte Carlo

arXiv:2608. 20123v1 Announce Type: cross Abstract: We study inference-time control for text generation in discrete diffusion language models, where the goal is to steer sampling toward sequence-level rewards without retraining.

arXiv Machine Learning
Jul 7

Non-Asymptotic Error Bounds for SMC with Biased Proposals: Application to Conditional Diffusion Sampling

arXiv:2607. 04780v1 Announce Type: cross Abstract: Sequential Monte Carlo (SMC) methods are a natural tool for post-hoc conditioning of pretrained generative models, but in many applications the mutation kernels used by the particle system are biased approximations of an ideal Feynman--Kac flow.

By Stanislas Strasman (SU, LPSM), Gabriel Victorino Cardoso (LPSM), Sylvain Le Corff (LPSM), Vincent Lemaire (LPSM), Antonio Ocello
arXiv Statistics ML
Oct 2

Error-Corrected Inference-Time Scaling for Imperfect Diffusion Models

The paper introduces the Energy-based Feynman-Kac Corrector (EBFKC), a method that improves inference-time scaling for diffusion models that are imperfect due to data and training limitations. EBFKC derives Feynman-Kac dynamics to exactly track a prescribed path in continuous time, then approximates these dynamics with sequential Monte Carlo and variance‑controlling guidance. Experiments on Gaussian mixtures, particle systems, and molecular simulations demonstrate that EBFKC closely matches target distributions and free‑energy profiles, outperforming standard inference‑time scaling baselines that still exhibit significant sampling errors.

By Zuokai Wen, Louis Grenioux, Weinan E, Jiequn Han
arXiv Computation and Language
Nov 6

SpecCTMC: Exact Speculative Simulation of Diffusion Language Model Continuous-Time Markov Chains

SpecCTMC is a training‑free speculative sampler for masked diffusion language models that reduces the sequential bottleneck of exact continuous‑time sampling. It updates tokens successively using previously computed predictions, evaluates intermediate states in parallel, and applies an exact acceptance‑and‑correction step, thereby preserving the joint distribution of event times, generation order, and token values. Experiments on LLaDA‑8B‑Instruct and Dream‑v0‑Instruct‑7B show 1.73‑4.38× fewer sequential model evaluations while maintaining the reference sampling distribution.

By Siva Rajesh Kasa, Yasong Dai, Sumit Negi, Hongdong Li
Hugging Face Trending Papers
Jul 26

Learning Sampling Parameters for Diffusion Models

Text-to-image diffusion models expose many inference-time sampling parameters, including prompts, negative prompts, classifier-free guidance scales, and noise schedules. These parameters are typically manually chosen once and then held fixed across prompts and denoising timesteps, even though different prompts and stages of generation can benefit from different parameter values.

arXiv Machine Learning
Aug 4

Just on Time: Token-Level Early Stopping for Diffusion Language Models

arXiv:2602. 11133v2 Announce Type: replace Abstract: Diffusion language models generate text through iterative refinement, a process that is often computationally inefficient because many tokens reach stability long before the final denoising step.

By Zakhar Kohut, Severyn Shykula, Mykola Vysotskyi, Serhii Dmytryshyn, Dmytro Khamula, Michal Zakrzewski, Damian Rynczak, Jacek Ma{\l}ecki, Taras Rumezhak, Volodymyr Karpiv