arXiv Computer Vision

DICE: Distilling Classifier-Free Guidance into Text Embeddings

arXiv Computer Vision
Sep 3

GlyphAnchor: Enhancing Visual Text Rendering via Position-Anchored Glyph Priors

GlyphAnchor is a new method that improves visual text rendering in image generation and editing models by adding lightweight glyph patch conditions anchored to the target image’s positional encoding. The approach is trained with staged supervised finetuning and text-aware post‑training, and it works with both text‑to‑image and image‑editing diffusion transformers. Experiments on various backbones and the newly introduced InfoTextBench benchmark show that GlyphAnchor consistently enhances text fidelity while maintaining overall image quality, especially for long, complex, or densely arranged text and rare characters.

By Qiang Xiang, Shuang Sun, Binglei Li, Yibo Chen, Xu Tang, Yao Hu, Junping Zhang
arXiv Computer Vision
Oct 16

Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation

Intern Lumina U2 is a diffusion large language model that uses multi-codebook visual tokens in a fully discrete space, allowing each visual position to be represented by a group of complementary codes. The model concatenates embeddings at each position and employs a lightweight autoregressive head to predict codes sequentially while the diffusion backbone processes all positions in parallel. Built on the unified 4D representation of AToken, it supports image, video, and 3D inputs, achieves leading performance in visual understanding, image generation, and editing, and includes a cross‑platform training, inference, and evaluation system with a stability‑aware sampling strategy that speeds up image generation by 4.45×.

By Qian Yu, Zongyang Qiu, Juncheng Yan, Hongliang Li, Qi Qin, Keqi Wang, Siqi Luo, Junkui Ju, Yapeng Mi, Tiantian Lu, Tian Liu, Pengyu Xie, Chenhao Ji, Minghui Zhang, Yiang Zhao, Siyu Yan, Haiying Xu, Yanbo Wang, Jianghan Shen, Yuandong Pu, Jiayi Lei, Borui Zhang, Jinxin Shi, Bo Zhang, Yuhang Zang, Shenglong Ye, Lixin Gu, Hongjie Zhang, Linghan Liu, Xinyuan Chen, Yuewen Cao, Lihe Zhang, Xiaohong Liu, Junjun He, Qiangwei Liu, Qi Zhang, Yu Cheng, Tianfan Xue, Hui Xiong, Chang Xu, Yu Qiao, Yihao Liu, Bin Fu
arXiv Machine Learning
Sep 25

CARE: Condition-Aware Representation Regularization for Diffusion Models

The paper introduces CARE, a lightweight, plug‑and‑play regularization framework for diffusion models that dynamically adjusts feature distributions based on condition similarity. By leveraging built‑in conditioning signals such as labels or text prompts, CARE promotes tighter feature clusters for similar conditions without requiring explicit alignment losses or external supervision. Empirical results show consistent improvements in visual fidelity and convergence stability, achieving significant FID reductions and speed‑ups on ImageNet and text‑to‑image tasks, and it can be combined with existing regularization methods for further gains.

By Fengjia Guo, Zhuoyi Yang, Jie Tang
arXiv AI
Oct 1

ReGain: Restoring Subject Fidelity in Personalization on Synthetic Images

ReGain is a training‑free correction that improves subject fidelity in text‑to‑image diffusion models personalized with synthetic images. The authors show that fine‑tuning on synthetic images degrades fidelity due to inflated classifier‑free guidance, especially at high frequencies. ReGain measures this inflation per frequency band and scales it down during sampling, closing 51‑64% of the fidelity gap on Stable Diffusion v1.5 and improving performance on SDXL and SD 3.5 while preserving text alignment.

By Shubhang Bhatnagar, Ishan Bhatnagar, Viraj Shah, Narendra Ahuja
arXiv Computation and Language
4d ago

ODC: Orthogonal Drift Correction for Improved Text-to-Image Semantic Alignment at Inference

The paper introduces Orthogonal Drift Correction (ODC), an inference-time guidance method that mitigates semantic drift in text-to-image generation without retraining or extra user input. ODC operates in two stages: first it detects drift by comparing the generated image to the prompt in a shared vision-language embedding space, isolates the orthogonal component of the drift, and converts it back into text; second it refines the text conditioning for a second generation pass using an adaptive rank-reduced concept removal module. Experiments show ODC improves prompt-image alignment, raising CLIP scores by up to 5.4% on COALBench-300 and PartiPrompts, and boosting the 3-in-1 score by 4.5% on T2I-CompBench’s complex subset.

By Kasra Kamali, Basim Azam, Tuan Duong Trinh, Jean Honorio, Naveed Akhtar