arXiv AI

Training-Free Hidden-State Refinement for Flow-Matching Image Generators

arXiv AI
1d ago

PEEK: One-Step-Look-Ahead Exposure Bias Correction for Diffusion Sampling

The paper introduces PEEK, a training‑free deterministic sampler plugin that corrects exposure bias in diffusion image generators by performing a provisional one‑step update and interpolating between predictions at a look‑ahead state. PEEK evaluates the denoiser at this higher‑signal noisy state, thereby producing more image‑like predictions and reducing the train‑test gap that often leads to blurry or poorly detailed outputs. Experiments on eight pixel‑space and latent‑space diffusion models—including unconditional, label‑guided, and text‑guided settings—show consistent improvements over NFE‑matched baselines, with FID gains up to 49.3 % and minimal additional computation.

By Yunwei Bai, Ying Kiat Tan, Tsuhan Chen
Hugging Face Trending Papers
Aug 6

Energy-Guided Flow Matching

Pixel-space generative models bypass lossy latent compression, yet necessitate joint learning of global structure and fine-grained details in a high-dimensional space. Standard flow matching interpolates noise toward a fixed clean-image endpoint, leaving the spectral evolution to be learned implicitly.

arXiv Machine Learning
Sep 3

CAT-Flow: Curvature-Adaptive sTeps for Flow Matching

The paper introduces CAT-Flow, a pair of lightweight, training‑free algorithms—CAT‑OV and CAT‑OT—that adapt step‑sizes during Flow Matching inference by estimating curvature in time or state space. These methods avoid extra neural evaluations and achieve constant‑order truncation error bounds. Experiments show that CAT‑OV and CAT‑OT improve image quality metrics across four text‑to‑image Flow Matching models, cutting the required generation steps by up to 40%.

By Qinchan Li, Pedro Cisneros-Velarde, Keru Fu, Samuel Antunes Miranda, Sharan Vaswani, Hao Zhang
arXiv AI
6d ago

How to Loop Your Transformer: Loop Scaling, Specialization, and Parallel Inference

The paper investigates how looping—reusing transformer weights to deepen models without adding parameters—affects performance under different constraints. Experiments on dense decoder‑only transformers up to 1 B parameters show that looping the full transformer yields best quality for a fixed parameter budget, while looping a middle block is preferable when training FLOPs are limited. Adding loop‑specific low‑rank adapters improves quality, but decoding slows linearly with loops; a parallel loop design that passes hidden states across token positions can mitigate this slowdown, though careful handling of token embeddings is needed to retain gains.

By Chanakya Ekbote, Artidoro Pagnoni, Devendra Singh Sachan, Shruti Bhosale, Mike Lewis, Maryam Fazel-Zarandi, Shashank Rajput, Sujan Kumar Gonugondla
arXiv Computer Vision
Sep 11

LoopVAE: Recurrent Depth Across Scales for Visual Tokenization

LoopVAE introduces a recurrent depth architecture that reuses a scale‑ and loop‑conditioned core across different spatial scales while keeping resolution‑changing transitions separate. The four‑block core applies 28 block operations per encoder or decoder, enabling a 29M‑parameter convolutional model to achieve 0.28 rFID and 32.54 dB PSNR on ImageNet‑256 with roughly 65% fewer parameters than comparable VAEs. Experiments with both convolutional and Transformer operators, as well as ablations on parameter sharing, demonstrate competitive image quality metrics and reveal how targeted loop interventions and truncation affect reconstruction quality and computational trade‑offs.

By Zhiying Lu
arXiv Machine Learning
Sep 11

Thinking with Looped Flows

The paper introduces looped flows, a new approach that trains looped models using local denoising objectives to overcome the difficulty of training early updates for future ones. By enforcing temporal association through progressively decreasing noise levels and shared noise, the method encourages recurrent states to transfer useful computation over time. Inference is framed as integrating the velocity of a probability flow parameterized by the learned denoiser, allowing the model to solve harder problems by allocating more computation and producing multiple valid predictions from different initial noise samples. Across six reasoning benchmarks, looped flows outperform prior state‑of‑the‑art looped models, achieving 58.8% accuracy on ARC‑AGI‑1 and 12.2% on ARC‑AGI‑2.

By Ayhan Suleymanzade, Chanhyuk Lee, Floor Eijkelboom, Nicholas M. Boffi, \.Ismail \.Ilkan Ceylan, Jinwoo Kim
arXiv Machine Learning
Sep 30

Improved Distributional Diffusion Models

arXiv:2609.37147v1 Announce Type: cross Abstract: Distributional Diffusion Models (DDMs) replace the standard mean-prediction denoiser with a \emph{distributional} denoiser trained via a scoring rule...

By Tommaso Martorella, Alexandre Galashov, Felix Krause, Stefan Andreas Baumann, Valentin De Bortoli, Arthur Gretton, Bj\"orn Ommer