The paper introduces ShadowBench, a benchmark built from real photographs that tests vision‑language models (VLMs) on three shadow‑related tasks: reading a visible shadow’s direction, inferring where a shadow should fall when absent, and judging physical consistency of shadows. Evaluations of 19 open VLMs and proprietary models show that while they can read shadow directions well, they perform poorly on inferring absent shadows and on consistency checks, with only Gemini‑3.1‑pro marginally above chance. The authors demonstrate that routing VLMs through image‑generation models (IGMs) significantly improves inference, suggesting that IGMs’ latent knowledge of illumination can bridge the gap between VLMs and physical reasoning.
By Ryugo Morita, Shunsuke Kitada, Stanislav Frolov, Brian Bernhard Moser, Ko Watanabe, Riku Takahashi, Jinjia Zhou, Andreas Dengel
Commercial vision-language models are reshaping computer vision, with visual priors broad enough to rival task-specific systems. This raises a natural question: do they reduce the need for classic, physics-informed low-level vision?
arXiv:2608. 06075v1 Announce Type: cross Abstract: Commercial vision-language models are reshaping computer vision, with visual priors broad enough to rival task-specific systems.
By Shilin Hu, Jingyi Xu, Dimitris Samaras, Hieu Le
OccludeBench is a manually curated benchmark that tests vision‑language models (VLMs) on their ability to recover fully occluded objects from remaining visual evidence. The benchmark distinguishes direct cases—requiring recovery from a specific physical cue such as a shadow or reflection—from inferential cases, which rely on broader scene context. Across five VLMs, accuracy averages 42.7 % on direct cases versus 62.7 % on inferential cases, a 20‑point gap, while humans achieve 63.4 % and 68.5 % respectively with no significant gap. The study introduces OccludeTrace, a four‑stage diagnostic that isolates the failure point, revealing that models struggle most when recovering the relevant object from the full image projection. Interventions such as inpaint‑then‑confront reduce the gap to a statistically non‑significant difference for models with image‑generation capabilities.
By Jasin Cekinmez, Addison J. Wu, Ryo Mitsuhashi, Linrong Cai, Xingyu Fu, Zhuang Liu
arXiv:2512.06174v3 Announce Type: replace
Abstract: Generating realistic cast shadows for inserted foreground objects requires reasoning about scene geometry and illumination. However, most learning-...
By Shilin Hu, Jingyi Xu, Akshat Dave, Dimitris Samaras, Hieu Le
arXiv:2606. 13156v2 Announce Type: replace-cross Abstract: Letting a vision-language model (VLM) think longer at test time has driven much recent progress.
By Animesh Tripathy, Aswanth Krishnan