arXiv AI

SQuID: A Benchmark for Quantitative Spatial Reasoning on Satellite Imagery

SQuID is a benchmark dataset comprising 2,000 quantitative spatial reasoning questions over 587 satellite images, covering 24 question types across three difficulty tiers. Answers are derived from curated segmentation masks or human consensus, with numeric responses scored against tolerance ranges based on human annotations. The benchmark also provides 75,687 training question-answer pairs and reference masks, enabling modular evaluation and highlighting segmentation as a key error source for current vision‑language models.

arXiv AI
Oct 31

ClueTracer: Question-to-Vision Clue Tracing for Training-Free Hallucination Suppression in Multimodal Reasoning

ClueTracer is a training‑free, parameter‑free plugin that suppresses hallucinations in multimodal reasoning models by tracing how visual clues propagate from the question through the model’s reasoning pathway. It localizes task‑relevant image patches while ignoring irrelevant regions, thereby addressing the problem of reasoning drift where models over‑focus on question‑irrelevant entities. The method improves performance across various reasoning architectures by an average of 1.21× on reasoning benchmarks.

By Gongli Xi, Kun Wang, Zeming Gao, Huahui Yi, Haolang Lu, Qianshan Wei, Ye Tian, Wendong Wang
Hugging Face Trending Papers
Aug 3

SpatialQuery: Benchmarking Geometry-Grounded Multi-Instance Spatial Reasoning in Vision-Language Models

Vision-language models (VLMs) achieve strong semantic understanding but remain unreliable in metric spatial reasoning, particularly when queries require comparing multiple instances of the same object category. We study this problem through the Closest-Instance Distance Query (CIDQ), where a model must identify the nearest visible candidate to a unique reference object and estimate their gravity-aligned floor-plane distance.

arXiv Computer Vision
Oct 12

TerraAgent: Demonstration-Grounded Multi-Agent Reasoning for Long-Context Remote Sensing Visual Question Answering

The paper introduces TerraAgent, a training‑free framework that enhances remote sensing visual question answering by integrating three modules: GDR for demonstration retrieval, MACE for evidence‑typed sub‑question decomposition, and CVA for counterfactual evidence auditing. It demonstrates significant improvements on multiple benchmarks, raising accuracy, visual‑dependency scores, and reducing prior‑only answers and hallucinations without fine‑tuning.

By Jingwen Huang, Yiming Fang, Hanxiao Wu, Boyuan He, Keyi Luo
arXiv AI
Sep 25

GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning

GeoRefer-Bench is a new benchmark for verifiable geospatial referring segmentation that evaluates whether models correctly resolve spatial relations in overhead imagery. Each query is expressed as an executable logical form over a metric scene graph, and predictions are scored with Exact Query Success (EQS), requiring an exact match to the query’s referent set. The dataset contains 700 UAV scenes, 26,217 instances, 142,796 spatial relations, 20,916 executable queries across five reasoning levels, and additional paraphrases, unanswerable queries, counterfactual pairs, and leakage‑controlled splits.

By Shuaishuai Cao, Min Huang, Meng Tang, Xuan Liu, Youjin Wang, Hui Lin
arXiv Computation and Language
Oct 28

PRISM-BENCH: Diagnosing Grounded and Faithful Multimodal Reasoning Beyond Final-Answer Accuracy

PRISM-BENCH is a new benchmark comprising 1,000 expert‑curated vision‑language questions across five under‑represented reasoning categories, each paired with an idealized reasoning trace validated by a second annotator. It introduces diagnostics beyond final‑answer accuracy: a perception probe that attributes about 85% of frontier‑model failures to misperception, and trace alignment that predicts answer correctness with a macro AUROC of 0.66. The benchmark also presents CORE, an efficiency‑aware metric that reorders 24.2% of model pairs relative to accuracy while remaining uncorrelated with response length, revealing that recent gains are driven by recognition rather than spatial grounding.

By Vansh Bhatia, Mehdi Rezagholizadeh, Parsa Ashrafi Fashi, Igor Gilitschenski, Vikram Appia, Emad Barsoum
arXiv AI
Aug 13

Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System

arXiv:2608. 11738v1 Announce Type: cross Abstract: Multimodal Large Language Model (MLLM)-based UAV aerial image understanding and reasoning is essential for aerial intelligence yet poses distinct challenges arising from extreme scale variation, arbitrary camera orientations, and high object density.

By Haoyu Zhang, Shuoxun Zhang, Peng Ye, Lin Zhang, Jiakang Yuan, Shenghong Yi, Yuening Wang, Tao Chen
arXiv Computer Vision
Sep 25

GeoNLI - A Natural Language Interpreter for Satellite Imagery

GeoNLI introduces a unified, modular pipeline that combines advanced SAM variants with multimodal large language models to perform satellite image captioning, visual question answering (VQA), and visual grounding. The EarthMind model achieves strong results on captioning and VQA, while multiple RemoteSAM-SAM and DiffuSAM pipelines are used for grounding, ultimately employing a majority‑voting ensemble across several models. The system reports 82% captioning accuracy, 83.32% VQA accuracy, and 64.94% grounding accuracy, demonstrating improved consistency over task‑specific approaches.

By Ashutosh Gandhe, Anupam Rawat, Geet Sethi, Kabir Nasiruddin, Madhav Kotecha, Panav Shah, Rakshit Sawarn, Soumitra Nayak
arXiv Computer Vision
Aug 25

WADE: A Reasoning-Annotated Benchmark for Multi-Instance Floating-Waste Grounding with Compact Vision-Language Models

arXiv:2608.22950v1 Announce Type: new Abstract: Floating waste in inland waterways threatens aquatic ecosystems and requires timely monitoring under cluttered, multi-object conditions. Existing aquat...

By Md. Asaduzzaman Shuvo, Ahsan Farabi, Md. Abdul Ahad Minhaz, Mahedi Hasan, Israt Khandaker, Ibrahim Khalil Shanto, Muhammad Nomani Kabir