arXiv AI
Jul 24
From Atoms to Entropy: Optimal Noise Allocation for Diffusion Training in the Convex Regime
arXiv:2607. 20540v1 Announce Type: cross Abstract: How should a diffusion model decide which noise levels to train on, and how much?
Geometric and Stochastic Analysis of Discontinuities in Sparse Mixture-of-Experts
arXiv:2606. 19036v1 Announce Type: new Abstract: Sparse Mixture-of-Experts (SMoE) architectures are now widely deployed in state-of-the-art language and vision models, where conditional routing allows scaling to very large networks.
Related stories
arXiv AI
Jun 29
Trust Region Masking for Long-Horizon LLM Reinforcement Learning
arXiv:2512. 23075v5 Announce Type: replace-cross Abstract: Policy gradient methods for Large Language Models optimize a policy $\pi_\theta$ via a surrogate objective computed from samples of a rollout policy $\pi_{\text{roll}}$.
arXiv AI
Jun 2
ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts
arXiv:2606. 01509v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models scale by activating only a small subset of experts per token.
arXiv Machine Learning
Jul 14
Sharp Concentration Bounds for Bundle-Valued Statistics on Manifolds
arXiv:2607. 10592v1 Announce Type: new Abstract: Many geometric statistics and manifold learning pipelines routinely produce observations -- such as tangent vectors or local frames -- whose natural home is a varying family of fibers attached to different points of a base manifold, rather than a single shared vector space.
arXiv Machine Learning
Aug 3
Hierarchical Copula-Gumbel-Top-\texorpdfstring{$K$}{K} Routing: Two-Sided Dependence Control for Frozen Mixture-of-Experts at Fixed Per-Token Routing Laws
arXiv:2607. 28670v1 Announce Type: new Abstract: A stochastic Gumbel-Top-$K$ router defines, for every token of a mixture-of-experts (MoE) model, a \emph{routing law}: a distribution over ordered expert lists and mixture weights.
arXiv AI
Jun 3
DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training
arXiv:2512. 13996v3 Announce Type: replace Abstract: Sparse Mixture-of-Experts architectures are essential for scaling model capacity efficiently, yet the standard Top-$k$ routing imposes a rigid sparsity pattern that ignores the intrinsic variance in token difficulty and layer-specific computational needs.
arXiv Machine Learning
Jun 25
Structured Approximations of Measures
arXiv:2310. 09149v3 Announce Type: replace-cross Abstract: We study the approximation of probability measures in the Wasserstein-$p$ distance by structured classes of approximators, motivated by applications in imaging, machine learning, and physical measurement under sensor constraints.
arXiv Machine Learning
Jul 30
Denoising growth complexity: Data geometry and certified schedules for diffusion sampling
arXiv:2607. 26285v1 Announce Type: cross Abstract: Two central challenges in diffusion-based sampling are the theoretical one of understanding their remarkable effectiveness even in high-dimensional settings, and the practical one of designing algorithms with certified performance guarantees.
arXiv Machine Learning
Jul 10
Expressivity and Statistical Trade-offs in Diffusion Policy Learning
arXiv:2607. 07967v1 Announce Type: cross Abstract: Diffusion-based policies have recently emerged as powerful policy parameterizations for reinforcement learning, representing state-conditioned action distributions as terminal laws of diffusion processes with parameterized drifts.
arXiv Machine Learning
Jul 1
Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees
arXiv:2602. 15008v2 Announce Type: replace Abstract: Diffusion models over discrete spaces have recently shown striking empirical success, yet their theoretical foundations remain incomplete.
arXiv Machine Learning
Jun 2
A Doeblin-Anchored Contrastive Chart for Learning Markov Transition Kernels
arXiv:2606. 02232v1 Announce Type: new Abstract: Learning a Markov transition model is not merely conditional density estimation: the learned object must be a valid transition kernel before it is iterated in downstream dynamics.
arXiv Machine Learning
Jun 19
Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures
arXiv:2606. 19894v1 Announce Type: new Abstract: The remarkable success of score-based diffusion models has spurred significant efforts to establish their theoretical foundations.