Hugging Face Trending Papers

Stage-Aware Adaptation and Distribution Calibration for Subject-Driven Personalized Text-to-Image Generation

Subject-driven personalized text-to-image generation requires a pretrained diffusion model to acquire a specific subject from a few reference images while preserving subject identity, following novel text prompts, and maintaining sample diversity. Existing optimization-based methods instantiate subject adaptation through full fine-tuning, textual embedding optimization, or low-rank parameter updates; PaRa further constrains personalization from the perspective of parameter rank reduction.

arXiv AI
Aug 20

From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model

arXiv:2608. 18339v1 Announce Type: cross Abstract: Vision-language models (VLMs) have demonstrated remarkable zero-shot capabilities yet remain sensitive to real-world distribution shifts during inference.

By Qi Yu, Zhichen Zeng, Katherine Tieu, Xiyuan Yang, Ruizhong Qiu, Yuchen Yan, Lihui Liu, Yanjun Zhao, Lingjie Chen, Jingrui He, Hanghang Tong
Hugging Face Trending Papers
Jun 1

Towards Resolving Optimization Conflicts Between Image- and Text-Based Person Re-Identification

The joint optimization of image-based (I2I) and text-based (T2I) person re-identification (ReID) is hindered by modality discrepancies and conflicting training objectives, leading to suboptimal shared representations. While I2I ReID focuses on identity-level invariance across images of the same person, T2I ReID is driven by instance-specific textual descriptions tied to unique visual traits.

arXiv Computer Vision
Aug 25

VISTA: Test-Time Compositional Alignment for Visual Autoregressive Generation

VISTA is a gradient‑based test‑time alignment framework designed for next‑scale visual autoregressive (VAR) image generation. It optimizes intermediate representations within the frozen transformer to enforce compositional constraints, without altering model weights or requiring extra training. Experiments on two benchmarks and two model scales show that VISTA improves compositional accuracy by up to 20% on a 2B backbone and 6% on an 8B backbone, while preserving image quality and enabling a smaller model to outperform a larger one.

By Hossein Shahabadi, Niki Sepasian, Mahdieh Soleymani Baghshah
arXiv AI
Sep 3

Inference-Time Optimization of Prompt Embeddings in Diffusion Models: A Comparison of sep-CMA-ES and Adam

The paper investigates inference‑time optimization of prompt embeddings for the Stable Diffusion XL Turbo model, comparing the gradient‑free Separable Covariance Matrix Adaptation Evolution Strategy (sep‑CMA‑ES) with the gradient‑based Adam optimizer. Using a weighted objective that blends LAION Aesthetic Predictor V2 and CLIPScore, the study evaluates 36 prompts under three weighting regimes (aesthetics‑only, balanced, alignment‑only). Across all settings, sep‑CMA‑ES consistently outperforms Adam in objective value, while also demonstrating favorable divergence metrics and lower compute and memory footprints, indicating its effectiveness as an inference‑time optimizer without requiring model fine‑tuning.

By Dom\'icio Pereira Neto, Jo\~ao Correia, Penousal Machado