The paper introduces ShadowBench, a benchmark built from real photographs that tests vision‑language models (VLMs) on three shadow‑related tasks: reading a visible shadow’s direction, inferring where a shadow should fall when absent, and judging physical consistency of shadows. Evaluations of 19 open VLMs and proprietary models show that while they can read shadow directions well, they perform poorly on inferring absent shadows and on consistency checks, with only Gemini‑3.1‑pro marginally above chance. The authors demonstrate that routing VLMs through image‑generation models (IGMs) significantly improves inference, suggesting that IGMs’ latent knowledge of illumination can bridge the gap between VLMs and physical reasoning.
By Ryugo Morita, Shunsuke Kitada, Stanislav Frolov, Brian Bernhard Moser, Ko Watanabe, Riku Takahashi, Jinjia Zhou, Andreas Dengel
Commercial vision-language models are reshaping computer vision, with visual priors broad enough to rival task-specific systems. This raises a natural question: do they reduce the need for classic, physics-informed low-level vision?
arXiv:2608. 06075v1 Announce Type: cross Abstract: Commercial vision-language models are reshaping computer vision, with visual priors broad enough to rival task-specific systems.
By Shilin Hu, Jingyi Xu, Dimitris Samaras, Hieu Le
OccludeBench is a manually curated benchmark that tests vision‑language models (VLMs) on their ability to recover fully occluded objects from remaining visual evidence. The benchmark distinguishes direct cases—requiring recovery from a specific physical cue such as a shadow or reflection—from inferential cases, which rely on broader scene context. Across five VLMs, accuracy averages 42.7 % on direct cases versus 62.7 % on inferential cases, a 20‑point gap, while humans achieve 63.4 % and 68.5 % respectively with no significant gap. The study introduces OccludeTrace, a four‑stage diagnostic that isolates the failure point, revealing that models struggle most when recovering the relevant object from the full image projection. Interventions such as inpaint‑then‑confront reduce the gap to a statistically non‑significant difference for models with image‑generation capabilities.
By Jasin Cekinmez, Addison J. Wu, Ryo Mitsuhashi, Linrong Cai, Xingyu Fu, Zhuang Liu
arXiv:2512.06174v3 Announce Type: replace
Abstract: Generating realistic cast shadows for inserted foreground objects requires reasoning about scene geometry and illumination. However, most learning-...
By Shilin Hu, Jingyi Xu, Akshat Dave, Dimitris Samaras, Hieu Le
arXiv:2606. 13156v2 Announce Type: replace-cross Abstract: Letting a vision-language model (VLM) think longer at test time has driven much recent progress.
By Animesh Tripathy, Aswanth Krishnan
The paper introduces a counterfactual benchmark to evaluate whether vision models understand physical constraints or merely learn rendering shortcuts. Using Mitsuba 3, it generates 5,500 synthetic images across 500 scene families, each with a clean image and ten violations (shadows, reflections, support, surface response, occlusion). Diagnostic baselines (LLaVA‑1.5‑7B, Qwen2.5‑VL‑7B, InternVL3.5‑8B) achieve 64.0–67.8 % macro‑F1 on a held‑out synthetic test set, but performance drops to 40.8 % for LLaVA‑1.5‑7B under an intervention shift, indicating reliance on rendering cues rather than true physical reasoning.
By M. Moein Esfahani, Sepehr Salem, Mohammed Alser, Vince Calhoun
The paper introduces a free, label‑free visual evidence signal that improves fine‑grained vision‑language reasoning. By selecting image crops that maximize the model’s answer distribution peak, the method locates answer‑bearing regions without training or annotations, boosting accuracy from 70 % to 85 %. The evidence gap also complements model confidence, enabling better correctness prediction and error flagging.
By Santi Ram Tiwari, Nihal Naik, Devbrat Pandey, Nishant Sinha
The paper introduces BedBench, a benchmark of thirteen procedurally generated tasks designed to test vision‑language models’ ability to recover and consistently use exact 3D object properties. Across twelve open‑weight and four frontier models, the study finds a stark gap: humans achieve 93.3 % accuracy on per‑color counting, while the best model scores only 7.2 %. Even targeted fine‑tuning can raise a model’s accuracy to 68.6 % but leaves its answers about the most common colors inconsistent with its own per‑color counts, highlighting that recovering a property does not guarantee its consistent use.
By Khai Le-Duc, Nguyen Kim Hai Bui, An Ngo, Nam D. Hoang, Yueqi Song, Jiarui Liu, Thai Quoc Hoang, Fang Wu, Chris Ngo, Terry Jingchen Zhang, Mona T. Diab, Ran Xu, Silvio Savarese, Xiao-Ming Wu
arXiv:2606.26738v3 Announce Type: replace
Abstract: While recent advancements in generative image editing models have achieved stunning visual fidelity, it remains an open question whether these syst...
By Tim K\"uchler, Johann-Friedrich Feiden, Matthias Nie{\ss}ner, Carsten Rother
arXiv:2607. 15565v1 Announce Type: cross Abstract: Where should the question go in a vision-language model (VLM) prompt: before the image or after it?
By Rakshanda Hassan Abhinandan, John Galeotti, Deva Ramanan, Gautam Rajendrakumar Gare
The paper introduces visual adaptations of counterfactual tests—vCT and vCCT—to evaluate whether chain-of-thought explanations in vision‑language models faithfully reflect the visual evidence driving predictions. Using these tests, the authors benchmark eight open‑source VLMs on two datasets and find that CoTs often fail to track visual evidence, sometimes omitting removed objects or mentioning them inconsistently. They also release two new datasets, Counter‑SNLI‑VE and Counter‑A‑OKVQA, consisting of image pairs that differ by a single object to facilitate further research.
By Bayar Menzat, Maximilian S\"uss, Ruizhi Wang, Benno Steinegger, Thomas Lukasiewicz, Oana-Maria Camburu