arXiv Machine Learning

Training Energy-Based Models with Non-MCMC Samplers and Efficient Temperature Estimation

The paper introduces Langevin simulated bifurcation (LSB), a fast, parallel Boltzmann sampler that matches the accuracy of sequential MCMC methods. It also proposes conditional expectation matching (CEM), an efficient technique for estimating the effective temperature of samples from energy‑based models with conditional independence. Building on these, the authors develop sampler adaptive learning (SAL), which adjusts the model temperature to align with the distribution produced by LSB, enabling efficient training of semi‑restricted Boltzmann machines (SRBMs) and outperforming conventional methods on synthetic spin‑glass datasets.

Hugging Face Trending Papers
Jul 29

Equilibrium Training of Energy-Based Models with Parallel Trajectory Tempering

Energy-Based Models (EBMs) provide an interpretable framework for generative modeling of scientific data, but poor Markov Chain Monte Carlo mixing often limits their reliability. We introduce a training algorithm based on Parallel Trajectory Tempering (PTT), which exploits the continuity of the optimization path to maintain equilibrium sampling throughout learning.

arXiv Machine Learning
Aug 19

Nonlocal Transition Kernel for Efficient Learning of Restricted Boltzmann Machines

The paper introduces a new transition kernel for Restricted Boltzmann Machines that operates over the sequence of models used in Deep Tempering. This kernel employs a round‑trip structure, allowing nonlocal moves in a single transition while keeping the RBM sequence unchanged. Experiments demonstrate that it achieves higher sampling quality with fewer transitions than both blocked Gibbs sampling and Deep Tempering, and it stabilizes learning by reducing training failures.

By Kaiji Sekimoto, Muneki Yasuda
arXiv Machine Learning
Aug 19

Composing Flow-Matching Energies with Known Physics: Generation, OOD Detection, and Inversion on PDE Fields

The paper presents a method that combines flow‑matching models with energy‑based modeling to explicitly construct scalar energy functions for physical fields. These energies are derived from a matching regression objective on a linear Gaussian interpolation, avoiding variational approximations or extra MCMC steps, and can be used for energy‑corrected data generation, out‑of‑distribution detection, and posterior sampling in inverse problems. The approach enables general MCMC samplers that reduce PDE residuals and spectral distance, and it demonstrates that combining data‑driven and physics‑based energies improves OOD detection accuracy.

By Yixuan Sun, Anirban Samaddar, Sandeep Madireddy
arXiv AI
3d ago

IsingFormer: Augmenting Parallel Tempering With Learned Proposals

The paper introduces IsingFormer, a Transformer model trained on long‑run MCMC configurations, which provides global proposal moves for Parallel Tempering (PT). By integrating these learned proposals into PT—forming Transformer‑Augmented Parallel Tempering (TAPT)—the authors demonstrate lower residual energies on 3D spin‑glass instances and improved efficiency on integer factorization tasks. A scaling study shows TAPT reduces the time‑to‑solution exponent by about 33% compared to standard PT across tested problem sizes.

By Saleh Bunaiyan, Corentin Delacour, Shuvro Chowdhury, Kyle Lee, Abdelrahman S. Abdelrahman, Kerem Y. Camsari