arXiv Computer Vision
LegendBench: A Diagnostic Benchmark for Legend Understanding with Counterfactual Interventions
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Sep 15
ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation
arXiv:2608.03464v2 Announce Type: replace Abstract: Annotations are essential to communicative visualization, helping explain data, emphasize key findings, and guide attention. While multimodal large...
arXiv Computation and Language
Sep 15
Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth
The paper investigates how vision‑language models (VLMs) read exact values from vertical bar charts. Using controlled counterfactual activation patching on Qwen2.5VL‑7B‑Instruct and InternVL3.5‑8B, the authors find that the bar‑top region contributes more to answer accuracy than the bar body, and that legend and series states lose recoverability earlier than geometry and axis‑scale states. They also show that prompt‑series positions mediate legend influence and that the two models differ in how they combine geometry and scale information from separate image donors.
arXiv AI
Jul 21
Seeing What Is Actually There: PriVE-Bench and PriVE-Tools for Counterfactual Evaluation of Agentic Visual Evidence in VLMs
arXiv:2607. 16311v1 Announce Type: cross Abstract: Vision-language models (VLMs) often answer visual questions using learned language and category priors rather than grounding their predictions in the image itself.
arXiv AI
Jul 28
Chart Deception in Vision-Language Models: From Vulnerability to Mitigation
arXiv:2607. 22600v1 Announce Type: new Abstract: Information visualizations are widely used to communicate patterns, trends, and outliers, yet deceptive design choices-such as truncated or inverted axes, distorted aspect ratios, inappropriate encodings, and misleading color mappings-can systematically alter interpretation while preserving the underlying data.
arXiv AI
Aug 10
SABRE: Scalable and Automated Benchmarking of VLMs under Stress
arXiv:2608. 07435v1 Announce Type: cross Abstract: Vision-language models (VLMs) are improving rapidly, but benchmark development lags behind, making weaknesses hard to identify.
arXiv AI
Sep 17
EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing
EDCT-Bench is a benchmark that evaluates the faithfulness of Vision‑Language Models (VLMs) by using Explanation‑Driven Counterfactual Testing (EDCT). EDCT extracts visual concepts from a model’s natural language explanation, applies minimal verified edits to those concepts, and checks whether the model’s answer and explanation remain consistent with the edited image. The benchmark covers three domains—knowledge‑intensive VQA, safety‑critical driving, and 3D spatial reasoning—and reveals significant faithfulness gaps in current VLMs, while also showing that EDCT‑generated counterfactuals can improve training.