arXiv Computer Vision
Sep 24

GLOW: Global Illumination-Aware Inverse Rendering of Indoor Scenes Captured with Dynamic Co-Located Light & Camera

GLOW is a Global Illumination‑aware inverse rendering framework for indoor scenes captured with dynamic co‑located light and camera setups. It combines a neural implicit surface representation with a neural radiance cache to jointly optimize geometry and reflectance, while introducing a dynamic radiance cache and a surface‑angle‑weighted radiometric loss to handle near‑field motion, strong inter‑reflections, and specular highlights. Experiments demonstrate that GLOW significantly outperforms prior methods in estimating material reflectance under both natural and co‑located illumination.

By Jiaye Wu, Saeed Hadadan, Geng Lin, Peihan Tu, Auguste Gezalyan, Matthias Zwicker, David Jacobs, Roni Sengupta
arXiv Computer Vision
Sep 11

Shedding Light: A Benchmark for Evaluating Lighting Understanding in Generative Image Models

The paper introduces a benchmark called Shedding Light to evaluate how well generative image models understand and reproduce lighting. The benchmark tests models by asking them to inpaint a simple object, called a light probe, into real photographs and then compares the generated probe to the ground truth to assess lighting direction, colour, and radiance. The authors provide a scalable protocol and open-source code and data for systematic assessment of photometric accuracy in future models.

By Justine Giroux, Jack Oliver Hilliard, Yannick Hold-Geoffroy, Javier Vazquez-Corral, Jean-Fran\c{c}ois Lalonde
arXiv AI
Sep 10

RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting

RelightFormer is a feed‑forward generative Transformer that performs single‑ and multi‑view image relighting without explicit intrinsic property estimation. It incorporates a latent illumination module that injects target environment maps into spatial features via cross‑attention, and uses permutation‑invariant positional encodings to process unordered multi‑view inputs symmetrically. Trained on the large Laval Objaverse Dataset, the model achieves state‑of‑the‑art visual and photorealistic relighting quality, and demonstrates strong zero‑shot generalization across various relighting tasks.

By Hejun Wang, Jinxi Li, Junwei Jiang, Shiwei Mao, Hu Cheng, Shouwang Huang, Bo Yang
arXiv AI
Sep 21

Physically Based Rendering in the Latent Space

The paper introduces a method that applies physically based rendering (PBR) within the latent space of variational autoencoders used in image diffusion models. By modifying the rendering equation and using a differentiable renderer, the authors can generate latent maps that guide content creation with physically accurate lighting. The approach is trained on a single rendered image and then shown to generalize to changes in scene geometry, lighting, and camera viewpoint.

By Vuk Radovanovic, Vishesh Gupta, Adrien Gruson, Binh-Son Hua