arXiv Machine Learning

On Color Alignment in VAE Latent Spaces and Its Applications

arXiv AI
Sep 18

Paint-Anything: Unified Any-Color Control for Image Generation and Editing

Paint-Anything introduces a unified hex-prompt interface that allows users to specify any 24‑bit hex color for both image generation and editing. The method trains on a new Paint‑500K dataset created from real images with object grounding, perceptual color labeling, and editing‑pair synthesis, and supplements this with pure‑color anchors to address shadow‑induced color inaccuracies. Evaluated on the newly proposed Any Color Benchmark (ACBench), Paint‑Anything achieves significant improvements over the base FLUX.2‑4B model, boosting T2I and editing scores by 85.3 % and 28.3 % respectively, and outperforms competing methods on the CompColor metric.

By Ji Xie, Dewei Zhou, Xinyu Huang, Zhennan Chen, Xun Wang
Hugging Face Trending Papers
Aug 13

PixSDS: Why Latent SDS Makes Noisy Pixels

Score Distillation Sampling (SDS) enables text-to-3D generation by optimizing rendered images with a pretrained diffusion prior, but latent SDS often produces structured color artifacts and high-frequency texture noise. We identify a failure mode of latent SDS caused by VAE-induced pixel drift: the optimized image can move along pixel-space directions that are weakly constrained by the VAE encoder, so its latent representation remains clean and semantically meaningful while the image itself accumulates visible artifacts.

arXiv Computer Vision
Oct 2

DiDE:Direct Injection with Color-Texture DEcoupling for 3D Stylization

DiDE is a training‑free framework that enables independent control of color and texture when stylizing 3D assets generated by rectified flow‑based image‑to‑3D models. It leverages the overcomplete latent space of these models, partitioning channels so that texture occupies a small subset while color is encoded in a free subspace. Experiments on the newly collected Disen3D‑Bench benchmark demonstrate that DiDE outperforms existing 2D and 3D stylization methods in color fidelity, texture transfer, and content preservation.

By Tao Wu, Alexandra Gomez-Villa, Senmao Li, Yaxing Wang, Joost van de Weijer, Kai Wang
arXiv Machine Learning
Aug 20

Simple, Safe, and Overlooked: Reclaiming Sustainable Domain Generalization with Statistical Color Matching

The paper introduces Colorist, a data‑augmentation method that uses classical statistical color matching to generate domain‑shifted medical images. By applying global mean‑standard‑deviation matching in RGB space, Colorist creates structurally intact variations without neural networks, outperforming deep generative models in fidelity and color alignment. Across multiple medical imaging datasets, it boosts balanced accuracy by up to 9% over state‑of‑the‑art domain‑generalization regularizers and 13% over no augmentation, while reducing computational cost and preserving anatomical structure.

By Sebastian Doerrich, Francesco Di Salvo, Shyam Nandan Rai, Marco Lents, Christian Ledig
Hugging Face Trending Papers
Jul 7

From RGB Generation to Dense Field Readout: Pixel-Space Dense Prediction with Text-to-Image Models

Large-scale text-to-image models are attractive backbones for dense prediction because RGB generation pretraining learns rich semantic, structural, and geometric priors. Existing generative and editing approaches reuse these priors by casting dense prediction as target generation: annotations such as depth, normals, alpha mattes, masks, and heatmaps are encoded into an RGB-trained VAE latent space and decoded back as image-like targets.

arXiv AI
Sep 21

Physically Based Rendering in the Latent Space

The paper introduces a method that applies physically based rendering (PBR) within the latent space of variational autoencoders used in image diffusion models. By modifying the rendering equation and using a differentiable renderer, the authors can generate latent maps that guide content creation with physically accurate lighting. The approach is trained on a single rendered image and then shown to generalize to changes in scene geometry, lighting, and camera viewpoint.

By Vuk Radovanovic, Vishesh Gupta, Adrien Gruson, Binh-Son Hua
arXiv Computer Vision
Oct 5

ChromaGS: Text-Driven Semantic Editing of 4D Gaussian Avatars

ChromaGS is a real‑time, language‑guided method for color editing of animatable 3D Gaussian head avatars. It augments each Gaussian primitive with learned soft assignments to semantic regions and decomposes colors into region‑level base colors and Gaussian‑level residuals, enabling coherent color transfer while preserving fine details. A two‑stage language pipeline translates natural‑language instructions into target colors, supporting both absolute and relative adjustments without requiring retraining.

By Antonio Canela, Jordi S\`anchez-Riera