arXiv AI By Dom\'icio Pereira Neto, Jo\~ao Correia, Penousal Machado

Inference-Time Optimization of Prompt Embeddings in Diffusion Models: A Comparison of sep-CMA-ES and Adam

Read the original on arXiv AI →

The paper investigates inference‑time optimization of prompt embeddings for the Stable Diffusion XL Turbo model, comparing the gradient‑free Separable Covariance Matrix Adaptation Evolution Strategy (sep‑CMA‑ES) with the gradient‑based Adam optimizer. Using a weighted objective that blends LAION Aesthetic Predictor V2 and CLIPScore, the study evaluates 36 prompts under three weighting regimes (aesthetics‑only, balanced, alignment‑only). Across all settings, sep‑CMA‑ES consistently outperforms Adam in objective value, while also demonstrating favorable divergence metrics and lower compute and memory footprints, indicating its effectiveness as an inference‑time optimizer without requiring model fine‑tuning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Aug 25

VISTA: Test-Time Compositional Alignment for Visual Autoregressive Generation

VISTA is a gradient‑based test‑time alignment framework designed for next‑scale visual autoregressive (VAR) image generation. It optimizes intermediate representations within the frozen transformer to enforce compositional constraints, without altering model weights or requiring extra training. Experiments on two benchmarks and two model scales show that VISTA improves compositional accuracy by up to 20% on a 2B backbone and 6% on an 8B backbone, while preserving image quality and enabling a smaller model to outperform a larger one.

By Hossein Shahabadi, Niki Sepasian, Mahdieh Soleymani Baghshah
Hugging Face Trending Papers
Aug 3

MIEScore: Human-Aligned Evaluation for Multi-Source Image Editing

Recent advances in unified multimodal models have significantly improved text-guided image editing abilities. In particular, models such as Nano-Banana-Pro and GPT-Image-2 demonstrate emerging capabilities in multi-source image editing (MIE), including tasks such as object synthesis, person-background composition, and cross-image style fusion.

Hugging Face Trending Papers
Jul 8

Stage-Aware Adaptation and Distribution Calibration for Subject-Driven Personalized Text-to-Image Generation

Subject-driven personalized text-to-image generation requires a pretrained diffusion model to acquire a specific subject from a few reference images while preserving subject identity, following novel text prompts, and maintaining sample diversity. Existing optimization-based methods instantiate subject adaptation through full fine-tuning, textual embedding optimization, or low-rank parameter updates; PaRa further constrains personalization from the perspective of parameter rank reduction.