arXiv AI
Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 12
Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
arXiv:2608. 10954v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) demonstrate impressive performance in benign scenarios, their cognitive reliability deteriorates significantly in complex scenes under adverse conditions.
arXiv AI
Jun 17
m2sv: A Scalable Benchmark for Map-to-Street-View Spatial Reasoning
arXiv:2601. 19099v2 Announce Type: replace-cross Abstract: Vision--language models (VLMs) achieve strong performance on many multimodal benchmarks but remain brittle on spatial reasoning tasks that require aligning abstract overhead representations with egocentric views.
arXiv Computation and Language
Aug 31
Cognitive Chain-of-Thought (CoCoT): Structured Multimodal Reasoning about Social Situations
The paper introduces Cognitive Chain-of-Thought (CoCoT), a structured reasoning framework for vision‑language models that divides multimodal social reasoning into three cognitively inspired stages: Perception, Situation, and Norm. CoCoT improves performance across diverse tasks—multimodal intent disambiguation, theory of mind, social commonsense reasoning, and safety instruction following—by 5.9% to 4.6% on average. Fine‑tuning on CoCoT‑structured traces further boosts accuracy by 5–6% without explicit prompting, indicating that models internalize the structured reasoning pattern and that the approach enhances interpretability and social alignment in multimodal systems.
arXiv AI
Oct 9
Humanity's Sixth Sense: Benchmarking Intuitive Visual Reasoning in Multimodal Models
The paper introduces Humanity's Sixth Sense (HSS), a benchmark designed to evaluate intuitive visual reasoning in multimodal large language models. HSS covers a wide range of image and video inputs, organizes them under a structured taxonomy, and pairs each with human‑written prompts that probe implicit temporal, spatial, social, and abstract cues that humans can infer at a glance. Current state‑of‑the‑art models, including GPT‑6‑astra, achieve far lower accuracy than humans (53.6 % vs. 93.1 %), highlighting a significant gap in intuitive visual understanding.
arXiv Machine Learning
Jul 2
VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning
arXiv:2511. 17731v2 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) prompting has proven remarkably effective for eliciting complex reasoning in large language models (LLMs).
arXiv Computation and Language
2d ago
MagiC: Evaluating Multimodal Cognition Toward Grounded Visual Reasoning
arXiv:2507.07297v2 Announce Type: cross Abstract: Recent advances in large vision-language models have led to impressive performance in visual question answering and multimodal reasoning. However, it...