arXiv:2607. 00491v1 Announce Type: cross Abstract: Benchmarks for vision-language models (VLMs) mostly test observational spatial reasoning: models describe relations already visible in the input.
GroundSight at GroundLM 2026 Shared Tasks: GoldenViewVQA introduces a new VQA benchmark that requires models to answer driving‑scene questions while also identifying the camera view that supports the answer. The authors present CoVeR‑VQA, a training‑free, multi‑stage verification and correction framework that starts with GPT‑5.6 zero‑shot predictions and applies successive view‑specific, prior‑guided, and cross‑split group‑level verifications using Gemini‑3.6‑Flash, Claude‑Opus‑5, and semantic question grouping. On the official test set, CoVeR‑VQA achieves 84.75 % joint accuracy, surpassing the zero‑shot baseline by 13.56 percentage points, and the final run reaches 88.14 % joint accuracy after evaluator‑informed post‑hoc corrections.
The paper introduces the Answer-Evidence Support Protocol (AESP) to evaluate whether the grounding evidence in remote sensing visual question answering actually supports the model’s answer. It applies task‑specific rules for existence, counting, and comparison questions and is tested on the newly constructed G‑RSVQA benchmark, which contains 15,027 samples with detailed object‑level annotations. Experiments on seven grounded vision‑language models show that up to 50.82% of correct answers are not supported by the provided grounding evidence.
arXiv:2608. 08021v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) should answer from concrete image evidence rather than language priors, dataset shortcuts, or irrelevant visual context.
StateSight is a new benchmark designed to isolate and evaluate the ability of vision‑language models to reconstruct latent spatial structure from a single image. It consists of three procedurally generated task families—cube‑net opposite‑face reasoning, occluded cube‑tower counting, and 4‑neighbor connected‑component counting—each with 300 deterministic prompts and exact‑match scoring. The benchmark also includes a companion dataset, StateSight‑Steps, with 900 image‑text examples and 3,600 intermediate visual states to aid analysis of reconstruction errors.
The paper introduces three new vision‑centric evaluation benchmarks—temporal frame retrieval, video future prediction, and causal memory distortion—to assess visual question answering in large video models. Unlike traditional benchmarks that rely on text-based multiple choice questions, these tasks require models to reason directly from visual inputs. The authors find that current state‑of‑the‑art models struggle with visual queries, highlighting a gap in visual understanding that future research should address.
arXiv:2605.26380v2 Announce Type: replace-cross Abstract: Frontier multimodal large language models (MLLMs) have been reported to achieve over 90\% accuracy on fine-grained perception benchmarks. How...
Cross-image comparative reasoning remains challenging for vision-language models (VLMs), especially when correct prediction requires fine-grained attribute grounding and globally consistent reasoning. We present CoRe, a unified framework for this problem.
arXiv:2603.14733v2 Announce Type: replace Abstract: Multimodal Large Language Models have achieved strong performance in single-video understanding, yet their ability to reason across multiple videos...
arXiv:2604.01569v2 Announce Type: replace Abstract: Video multimodal large language models achieve strong results on existing benchmarks, but answer accuracy alone does not establish whether they can...
SpaceCast-Bench is a new benchmark that evaluates predictive spatial reasoning in vision‑language models, moving beyond simple spatial perception to tasks that require constructing scenes, anticipating interventions, and reasoning about unseen outcomes. It contains 3,862 questions from 182 real‑world scenes across 16 task types and three difficulty levels—static perception, local prediction, and global prediction—testing scene understanding, spatial state updating, and relational inference. Evaluation of 21 models shows a large performance gap, with the best model achieving only 58.0% versus 87.2% human accuracy, and fine‑tuning on the benchmark’s programmatically generated data can substantially improve model performance.
PRISM-BENCH is a new benchmark comprising 1,000 expert‑curated vision‑language questions across five under‑represented reasoning categories, each paired with an idealized reasoning trace validated by a second annotator. It introduces diagnostics beyond final‑answer accuracy: a perception probe that attributes about 85% of frontier‑model failures to misperception, and trace alignment that predicts answer correctness with a macro AUROC of 0.66. The benchmark also presents CORE, an efficiency‑aware metric that reorders 24.2% of model pairs relative to accuracy while remaining uncorrelated with response length, revealing that recent gains are driven by recognition rather than spatial grounding.