arXiv:2608. 25138v1 Announce Type: new Abstract: Stochastic masking, cropping, or modality removal makes deterministic reconstruction an incomplete target: one observation can admit many clean completions.
The information bottleneck (IB) trades compression of an input $X$ against prediction of a target $Y$ through a multiplier $β=1/λ$, yet each operating point is usually served by training, or fully con...
MUNITE is a latent‑variable framework that unifies encoding and latent generation for any‑to‑any multimodal generation. By treating different amounts of observed evidence as varying inference problems, it models the conditional distribution over a shared latent representation, using a single conditional flow model. The approach employs self‑distillation with conditional flow matching to learn from incomplete data, achieving competitive or superior generation quality and coherence across datasets such as PolyMNIST‑D‑Q, FFHQ64, and image‑text‑audio.
arXiv:2605. 12951v2 Announce Type: replace-cross Abstract: We propose Coreset-Induced Conditional Velocity Flow Matching (CCVFM), a generative model that augments hierarchical rectified flow with a data-informed source distribution.
arXiv:2607. 01799v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) decompose internal activations of neural networks into sparse linear combinations of learned features by fitting an overcomplete dictionary $\mathbf{W}\in\mathbb{R}^{m\times n}$ with $m<n$, and inferring a sparse code $\mathbf{x}\in\mathbb{R}^n$ from $\mathbf{h}\approx\mathbf{W}\mathbf{x}$.
The paper introduces Conditional Predictive Sufficient Statistics (CPSS) as a formal way to capture useful visual representations that preserve latent factors shared with future data while discarding noise. It shows that predicting the next image patch embedding with a cosine loss approximates maximum likelihood under a von Mises-Fisher model, and that stop‑gradient alone does not enforce sufficiency. Experiments on MNIST and CIFAR‑10 with small causal Transformers demonstrate that CPSS readouts outperform intermediate blocks, while removing stop‑gradient collapses embedding rank even when the pretext loss appears perfect.
The paper introduces the Velocity Ambiguity Profile (VAP), a time‑local Bayes‑risk floor for velocity prediction in flow matching, and decomposes the loss into an irreducible ambiguity term and a reducible excess term. It provides statistical benchmarks for intrinsic local‑regression and Gaussian‑mixture scenarios, and validates the diagnostic with controlled experiments and CIFAR‑100 DINOv2 feature flow, showing that guided time sampling can reduce velocity loss and Fréchet distance.
arXiv:2606. 29176v1 Announce Type: new Abstract: A deep network's loss is invariant to continuous symmetries of its parameters: the logit shift, the ReLU rescaling, the LayerNorm scale, the per-head attention rotation.
The paper introduces Untied Self-Conditioning, a sampler that corrects a train–inference mismatch in flow‑matching language models. By dampening redundant directions in the self‑conditioning input and approximating a step‑average prediction from history, the method improves generation quality without retraining. On LangFlow and ELF‑B datasets, it dramatically lowers perplexity and is preferred in the majority of pairwise comparisons.
A deep network's loss is invariant to continuous symmetries of its parameters: the logit shift, the ReLU rescaling, the LayerNorm scale, the per-head attention rotation. Adam's per-coordinate preconditioner drifts along each symmetry orbit, which pulls the trajectory off the symmetry quotient where the optimization lives and blurs the singular-learning rate the quotient makes readable.
arXiv:2607. 03809v1 Announce Type: new Abstract: Normalising flows provide a powerful variational family for approximate inference, yet individual architectures often fail to generalise across heterogeneous posterior geometries.
The paper challenges the common practice of selecting latent world models based on one‑step prediction loss, arguing that this metric poorly predicts planning performance. Using a frozen‑decoder protocol, the authors decompose one‑step error into encoder‑inherent and predictor‑added components and evaluate goal‑conditioned planning across three continuous‑control environments. Their experiments show that improvements in one‑step fidelity do not correlate with planning success, with correlation coefficients near zero and inconsistent exchange rates across tasks.
The paper studies generative flows that sample from an unnormalized target by training a flow to satisfy a balance condition. It shows that the training loss can certify sampler accuracy, can be driven to zero, and quantifies how fast gradient descent converges, with explicit bounds depending on the backward policy and graph structure. The authors provide theoretical guarantees, test them on finite graphs, and certify each theorem using a Lean 4 development.