The paper introduces ShadowBench, a benchmark built from real photographs that tests vision‑language models (VLMs) on three shadow‑related tasks: reading a visible shadow’s direction, inferring where a shadow should fall when absent, and judging physical consistency of shadows. Evaluation of 19 open VLMs and proprietary models shows that while they can read shadow direction well, they perform poorly on inferring absent shadows and judging consistency, with only Gemini‑3.1‑pro slightly above chance. The authors demonstrate that routing a VLM through an image‑generation model (IGM) provides the missing knowledge, significantly improving inference and highlighting the potential of image‑generation objectives for illumination‑aware VLMs.
Commercial vision-language models are reshaping computer vision, with visual priors broad enough to rival task-specific systems. This raises a natural question: do they reduce the need for classic, physics-informed low-level vision?
arXiv:2608. 06075v1 Announce Type: cross Abstract: Commercial vision-language models are reshaping computer vision, with visual priors broad enough to rival task-specific systems.
By Shilin Hu, Jingyi Xu, Dimitris Samaras, Hieu Le
OccludeBench is a manually curated benchmark that tests vision‑language models (VLMs) on their ability to recover fully occluded objects from remaining visual evidence. The benchmark distinguishes direct cases—requiring recovery from a specific physical cue such as a shadow or reflection—from inferential cases, which rely on broader scene context. Across five VLMs, accuracy averages 42.7 % on direct cases versus 62.7 % on inferential cases, a 20‑point gap, while humans achieve 63.4 % and 68.5 % respectively with no significant gap. The study introduces OccludeTrace, a four‑stage diagnostic that isolates the failure point, revealing that models struggle most when recovering the relevant object from the full image projection. Interventions such as inpaint‑then‑confront reduce the gap to a statistically non‑significant difference for models with image‑generation capabilities.
By Jasin Cekinmez, Addison J. Wu, Ryo Mitsuhashi, Linrong Cai, Xingyu Fu, Zhuang Liu
arXiv:2512.06174v3 Announce Type: replace
Abstract: Generating realistic cast shadows for inserted foreground objects requires reasoning about scene geometry and illumination. However, most learning-...
By Shilin Hu, Jingyi Xu, Akshat Dave, Dimitris Samaras, Hieu Le
arXiv:2607. 15565v1 Announce Type: cross Abstract: Where should the question go in a vision-language model (VLM) prompt: before the image or after it?
By Rakshanda Hassan Abhinandan, John Galeotti, Deva Ramanan, Gautam Rajendrakumar Gare