VisualTrust introduces a counterfactual benchmark to evaluate how multimodal large language models handle on‑screen claims that may conflict with visual evidence. The benchmark contains 5,063 cases across 14 claim‑fact classes, each featuring a question paired with three video versions: no claim, accurate claim, and misleading claim. Results show a drop in accuracy from 90.1% to 51.9% when a misleading claim is present, and a new training‑free agent, DEAR, improves accuracy under misleading claims to 81.8–85.2% while reducing trap‑answer selection.
arXiv:2608. 15574v1 Announce Type: cross Abstract: Video question answering systems built on vision-language models often produce timestamped claims with high confidence even when unsupported by the cited frame.
MiniVer‑V is a new benchmark for short‑video fact‑checking that contains 195 videos, each annotated with a supported, refuted, or insufficient verdict and 5,510 multimodal evidence units (visual keyframes, speech transcripts, and web‑retrieved sources). The authors propose a two‑layer verification framework that first checks claim‑video consistency using internal evidence, then determines the factual verdict with external corroboration, and employs a sufficiency‑driven greedy search to assemble only the minimal evidence needed for a confident decision. Using Claude Sonnet 4, the method achieves a Macro‑F1 of 0.510 with an average of 4.5 evidence units—about 16 % of the full set—while matching the full‑evidence baseline and improving detection of insufficient cases.
The paper introduces Mind2Web-Injection, a benchmark of 9,954 instruction‑screenshot pairs designed to evaluate whether vision‑language models (VLMs) use visual evidence when judging conflicts between on‑screen text and user instructions. Across six VLMs, models with similar overall precision can differ dramatically in Evidence‑Aligned Detection (EAD), revealing that verdict‑only metrics miss critical grounding failures. The authors also propose two training‑free interventions—ReadGate and CmdCompare—to improve evidence grounding and instruction‑command consistency, and argue for separate reporting of verdict correctness, evidence localization, and counterfactual responsiveness.
The study investigates how false content in training data can influence language models’ downstream decisions even without explicit triggers. By comparing models trained on misleading versus truthful documents in a controlled decision task and a real‑world bushfire case, the authors find a gap between factual answers and the decisions derived from them: correct facts do not always lead to correct decisions, and removing an injected number does not eliminate the misleading narrative. The research highlights that data poisoning can subtly alter model behavior beyond what is detectable through direct probing.
arXiv:2609.36572v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has been extended to Large Vision-Language Models (LVLMs), and perception-aware methods further e...
arXiv:2606. 26904v1 Announce Type: cross Abstract: Video reasoning language models implicitly assume that every input frame is equally reliable.
VisionQ is a new benchmark for qualitative analysis in computer vision that evaluates vision‑language models (VLMs) on criterion‑conditioned visual discrimination. It is built from over 1,800 peer‑reviewed comparison figures in CVPR and ICCV papers, linking each image crop to author‑stated visual claims through 3,911 hand‑annotated data points. The benchmark includes a 51‑leaf taxonomy of visual criteria, a protocol that hides method identities and reports accuracy per criterion, and a DPO‑tuned Gemma‑4‑E4B judge that improves accuracy on a held‑out test set.
The paper introduces EpiReal-Bench, a benchmark of 10,000 false-claim prompts and corresponding images designed to test visual misinformation risks in commercial image generators. It also presents EpiReal-Attack, a black-box optimization framework that can bypass alignment safeguards, increasing the success rate of generating misleading images from 70% to 95%. Experiments on four commercial models show that these generators can produce credible visual evidence for false claims, highlighting a blind spot in current alignment methods.
arXiv:2608. 13267v1 Announce Type: cross Abstract: Existing vision-language model (VLM) benchmarks emphasize perception and reasoning accuracy (how well VLMs describe and reason about what they see in an image), with limited attention to behavioral reliability under uncertainty (how they behave when visual evidence is missing or misleading).
ClaimScope is a 72‑task benchmark that audits mathematical claims by requiring a verdict, scope, counterexample, and repair, covering areas such as optimizer‑state transport and exact $2 imes2$ linear algebra. The study evaluates Qwen3‑8B using a witness checker that only examines the model’s submitted state, showing that verifier feedback improves counterexample acceptance and safe completions compared to generic feedback. The results demonstrate that separating claim classification from executable evidence can yield gains in witness‑structure correction without necessarily improving broader reasoning.
arXiv:2606. 07613v1 Announce Type: cross Abstract: Visual evidence has long been treated as a reliable form of legal proof, but advances in artificial intelligence (AI) are undermining that assumption.
arXiv:2610.16441v1 Announce Type: cross Abstract: Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Exp...