arXiv Machine Learning
WEIRDO: WEak resIdual Regularized DOob's h-transform diffusion alignment
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 1
Convergence rates for the RMSprop optimizer with full control of the hyperparameters
arXiv:2608.30382v1 Announce Type: new Abstract: Popular adaptive stochastic gradient descent (SGD) methods to train artificial intelligence (AI) systems include the RMSprop, the Adam, and the AdamW o...
arXiv Machine Learning
Aug 20
Sobolev Regularized Score Difference Estimation in Diffusion Models
arXiv:2608. 18237v1 Announce Type: cross Abstract: Estimating the difference of two Stein's score functions is a fundamental problem in generative modeling.
arXiv Machine Learning
Aug 12
Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime
arXiv:2603. 10485v3 Announce Type: replace-cross Abstract: In this work, we study the convergence properties of the Dual Space Preconditioned Gradient Descent, encompassing optimizers such as Normalized Gradient Descent and Gradient Clipping.
arXiv Machine Learning
Sep 23
Penalized Nonreversible Langevin for Constrained Sampling
The paper introduces penalized nonreversible Langevin algorithms for sampling from a target distribution constrained to a compact convex set. It combines a squared distance penalty with skew-symmetric perturbations that preserve the penalized Gibbs distribution, and provides nonasymptotic total variation and Wasserstein bounds under various smoothness and contraction assumptions. Numerical experiments demonstrate the methods on constrained Bayesian regression, classification, neural networks, and truncated sampling, highlighting acceleration in a stochastic quadratic model.
arXiv Machine Learning
Jul 27
On the Convergence of Stochastic Low-Rank Adaptation
arXiv:2607. 21975v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) optimizes $J(B,A)=\mathcal L(W_\mathrm{base}+sBA)$ over two adapters $B \in \mathbb{R}^{m \times r}$ and $A \in \mathbb{R}^{r \times n}$ that form a low-rank update to a frozen pretrained weight matrix $W_\mathrm{base} \in \mathbb{R}^{m \times n}$.
arXiv AI
Jun 19
Conditional Diffusion Guidance under Hard Constraint: A Stochastic Analysis Approach
arXiv:2602. 05533v3 Announce Type: replace Abstract: We study conditional generation in diffusion models under hard constraints, where generated samples must satisfy prescribed events with probability one.