arXiv Machine Learning
Aug 11
Path-dependent Discrete Amortized Inference
arXiv:2608. 08644v1 Announce Type: new Abstract: We consider the problem of sampling compositional and discrete objects from a given unnormalized posterior distribution.
Path-dependent Discrete Amortized Inference
We consider the problem of sampling compositional and discrete objects from a given unnormalized posterior distribution. Notably, recent studies have shown that this problem can be efficiently solved by learning a deterministic Markov Decision Process (MDP) that progressively builds each object in proportion to the posterior.
Related stories
arXiv Machine Learning
Sep 1
Delta-AI: Local objectives for amortized inference in sparse graphical models
arXiv:2310.02423v3 Announce Type: replace Abstract: We present a new algorithm for amortized inference in sparse probabilistic graphical models (PGMs), which we call $\Delta$-amortized inference ($\D...
arXiv Machine Learning
Aug 31
Improved off-policy training of diffusion samplers
The paper investigates training diffusion models to sample from distributions defined by unnormalized densities or energy functions. It benchmarks various diffusion-structured inference techniques, including simulation-based variational methods and off-policy approaches such as continuous generative flow networks, highlighting their relative strengths and challenging some prior claims. Additionally, the authors introduce a new exploration strategy for off-policy methods that employs local search in the target space with a replay buffer, demonstrating improved sample quality across multiple target distributions.
arXiv Machine Learning
Sep 1
Adaptive teachers for amortized samplers
arXiv:2410.01432v3 Announce Type: replace Abstract: Amortized inference is the task of training a parametric model, such as a neural network, to approximate a distribution with a given unnormalized d...
arXiv Machine Learning
Aug 31
Amortizing intractable inference in diffusion models for vision, language, and control
The paper introduces a data‑free learning objective called relative trajectory balance for training diffusion models to sample from a posterior defined by a diffusion prior and an arbitrary black‑box constraint or likelihood. It proves asymptotic correctness of this objective and demonstrates its use across vision, language, and multimodal tasks, including classifier guidance, language infilling, and text‑to‑image generation. Additionally, the method is applied to continuous control with a score‑based behavior prior, achieving state‑of‑the‑art results in offline reinforcement learning.
arXiv Machine Learning
Aug 3
Reinforced sequential Monte Carlo for amortised sampling
arXiv:2510. 11711v3 Announce Type: replace Abstract: This paper proposes a synergy of amortised and particle-based methods for sampling from distributions defined by unnormalised density functions.
arXiv AI
Jul 20
Energy-based Transport for Amortized Bayesian Inference
arXiv:2605. 15407v3 Announce Type: replace-cross Abstract: We consider amortized Bayesian inference for nonlinear inverse problems using only samples from the joint distribution of parameters and observations, including problems with unknown functions in a Banach space.
arXiv Machine Learning
Jun 25
Efficient Adaptive Data Acquisition via Pretrained Belief Representations
arXiv:2606. 25197v1 Announce Type: new Abstract: Learning effective policies for adaptive data acquisition remains challenging: posterior-based methods rely on surrogate models and posterior approximations that can be misspecified or biased, while direct policy-learning methods map from historical observations and fail to exploit available model representations, making learning harder.
arXiv Machine Learning
Sep 22
Thompson Sampling for Infinite-Horizon Discounted Decision Processes
arXiv:2405.08253v4 Announce Type: replace-cross Abstract: This paper develops a framework for learning in discounted infinite-horizon Markov decision processes (MDPs) with Borel state and action spac...
arXiv Machine Learning
Sep 7
Optimal Data Acquisition for Reinforcement Learning: A Large Deviations Perspective
The paper presents a large deviations framework for efficient data acquisition in infinite-horizon reinforcement learning, introducing the exponential decay rate of policy-selection error probability as a key efficiency metric. It derives a variational characterization leading to a nested optimization problem, then proposes a tractable convex relaxation and a lazy one-step projected subgradient method to construct an adaptive data acquisition policy. The resulting algorithm is shown to be near-robustly optimal under the proposed criterion, with extensions to linear function approximation and supporting numerical experiments.
arXiv Machine Learning
Aug 26
Model-Based Learning of Near-Optimal Finite-Window Policies in POMDPs
arXiv:2604.01024v2 Announce Type: replace Abstract: We study model-based learning of finite-window policies in tabular partially observable Markov decision processes (POMDPs). A common approach to le...
arXiv Machine Learning
Jun 16
Learning Policy from a Single Trajectory in Average-Reward Markov Decision Process
arXiv:2606. 16729v1 Announce Type: new Abstract: While there is an extensive body of work characterizing the sample complexity of discounted cumulative-reward MDPs, finite sample analyses for average-reward MDPs have been limited, and most existing works rely on restrictive assumptions such as ergodicity or access to a generative model.