arXiv AI By Dom\'icio Pereira Neto, Jo\~ao Correia, Penousal Machado

Inference-Time Optimization of Prompt Embeddings in Diffusion Models: A Comparison of sep-CMA-ES and Adam

Read the original on arXiv AI →

The paper investigates inference‑time optimization of prompt embeddings for the Stable Diffusion XL Turbo model, comparing the gradient‑free Separable Covariance Matrix Adaptation Evolution Strategy (sep‑CMA‑ES) with the gradient‑based Adam optimizer. Using a weighted objective that blends LAION Aesthetic Predictor V2 and CLIPScore, the study evaluates 36 prompts under three weighting regimes (aesthetics‑only, balanced, alignment‑only). Across all settings, sep‑CMA‑ES consistently outperforms Adam in objective value, while also demonstrating favorable divergence metrics and lower compute and memory footprints, indicating its effectiveness as an inference‑time optimizer without requiring model fine‑tuning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
4d ago

Rethinking the design space of Vanilla Pixel Diffusion Transformers

The paper revisits pixel-space diffusion models, showing that a vanilla diffusion transformer can generate directly in pixel space by swapping velocity prediction for clean image prediction. It introduces MFlow, an adaptive parameterization that interpolates between these two prediction modes based on data representation and model capacity, and demonstrates competitive performance on ImageNet, UCF‑101, and text‑to‑image tasks. Additionally, the authors add ConvRefiner, a lightweight convolutional module trained post‑hoc to restore high‑frequency details while preserving FID.

By Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava, Aniruddha Mahapatra
arXiv Computer Vision
Aug 25

VISTA: Test-Time Compositional Alignment for Visual Autoregressive Generation

VISTA is a gradient‑based test‑time alignment framework designed for next‑scale visual autoregressive (VAR) image generation. It optimizes intermediate representations within the frozen transformer to enforce compositional constraints, without altering model weights or requiring extra training. Experiments on two benchmarks and two model scales show that VISTA improves compositional accuracy by up to 20% on a 2B backbone and 6% on an 8B backbone, while preserving image quality and enabling a smaller model to outperform a larger one.

By Hossein Shahabadi, Niki Sepasian, Mahdieh Soleymani Baghshah