A new large-scale Arabic fact‑checking dataset called Arafa has been created using an automated pipeline that generates claims from Arabic Wikipedia, mutates them into counterfactuals, and validates them against supporting or refuting evidence. The dataset contains 181,976 claim‑evidence pairs labeled as supported, refuted, or not enough information, and human evaluation shows high inter‑annotator agreement and strong validation accuracy. Fine‑tuned transformer models on Arafa achieve a Macro F1‑score of 77%, demonstrating its usefulness for Arabic fact‑checking tasks.
By Christophe Khalil, Shady Elbassuoni, Rida Assaf
arXiv:2510.23508v4 Announce Type: replace
Abstract: Existing real-world datasets for multimodal fact-checking have multiple limitations: they contain few instances, cover only one or two languages, f...
By Jiahui Geng, Jonathan Tonglet, Iryna Gurevych
arXiv:2606. 24112v1 Announce Type: new Abstract: Multimodal misinformation detection is increasingly important because viral posts now combine long multilingual narratives, several images, mixed provenance, and subtle text--image framing errors.
By Chenhao Dang, Dantong Zhu, Jun Yang, Conghui He, Weijia Li
MiniVer‑V is a new benchmark for short‑video fact‑checking that contains 195 videos, each annotated with a supported, refuted, or insufficient verdict and 5,510 multimodal evidence units (visual keyframes, speech transcripts, and web‑retrieved sources). The authors propose a two‑layer verification framework that first checks claim‑video consistency using internal evidence, then determines the factual verdict with external corroboration, and employs a sufficiency‑driven greedy search to assemble only the minimal evidence needed for a confident decision. Using Claude Sonnet 4, the method achieves a Macro‑F1 of 0.510 with an average of 4.5 evidence units—about 16 % of the full set—while matching the full‑evidence baseline and improving detection of insufficient cases.
By Leran Chen, Lingnan Kong, Zile Cai
AfrIFact is a new dataset designed to support automatic fact‑checking in ten African languages and English, covering information retrieval, evidence extraction, and fact verification. Experiments show that current embedding models struggle with cross‑lingual retrieval, and that cultural and news documents are easier to retrieve than healthcare documents. Large language models also lack robust multilingual fact‑verification, but few‑shot prompting and task‑specific fine‑tuning can significantly improve performance.
By Israel Abebe Azime, Jesujoba Oluwadara Alabi, Crystina Zhang, Iffat Maab, Atnafu Lambebo Tonja, Tadesse Destaw Belay, Folasade Peace Alabi, Salomey Osei, Saminu Mohammad Aliyu, Nkechinyere Faith Aguobi, Bontu Fufa Balcha, Blessing Kudzaishe Sibanda, Davis David, Mouhamadane Mboup, Daud Abolade, Neo Putini, Philipp Slusallek, David Ifeoluwa Adelani, Dietrich Klakow
The paper introduces UrduFactBench and UrduFactQA, two hand‑annotated benchmarks for claim verification and factual consistency evaluation in Urdu, created through a multi‑stage annotation process with native speakers. It also presents UrduFactCheck, a modular fact‑checking framework that uses both monolingual and translation‑based evidence retrieval to address the scarcity of high‑quality Urdu evidence. Experiments on twelve LLMs show that translation‑augmented pipelines outperform monolingual ones, highlighting ongoing challenges for open‑source models in Urdu.
By Sarfraz Ahmad, Hasan Iqbal, Momina Ahsan, Numaan Naeem, Muhammad Ahsan Riaz Khan, Arham Riaz, Muhammad Arslan Manzoor, Yuxia Wang, Preslav Nakov
VisualTrust introduces a counterfactual benchmark to evaluate how multimodal large language models handle on‑screen claims that may conflict with visual evidence. The benchmark contains 5,063 cases across 14 claim‑fact classes, each featuring a question paired with three video versions: no claim, accurate claim, and misleading claim. Results show a drop in accuracy from 90.1% to 51.9% when a misleading claim is present, and a new training‑free agent, DEAR, improves accuracy under misleading claims to 81.8–85.2% while reducing trap‑answer selection.
By Huachao Zhu, Junjielong Xu, Sicheng Li, Liang Yang, Mingtao Chen, Zhichao Hu, Li Jiang, Pinjia He
The paper introduces Fact-Ablated Evaluation (FAE), a framework that iteratively removes cited evidence to test whether large language models (LLMs) adjust their fact‑checking predictions accordingly. Experiments reveal that many off‑the‑shelf LLMs rely more on internal knowledge than on the provided evidence. To address this, the authors propose REAL, a training method that uses counterfactual evidence supervision to encourage LLMs to base veracity judgments on evidence, achieving better evidence‑dependent performance across four datasets.
By Xingyu Deng, Mingzi Cao, Nikolaos Aletras, Xi Wang, Mark Stevenson
The paper introduces VeriSpeak, a benchmark of 3,879 spoken claims for evaluating fact verification in Large Audio Language Models (LALMs). It shows a clear modality gap: models that verify written claims well often fail on spoken versions, and retrieval alone offers limited improvement. Combining retrieval with explicit reasoning yields the best performance, reaching 86.1% accuracy and demonstrating the need for grounded reasoning over retrieved evidence in speech misinformation detection.
By Debajyoti Mazumder, Mamta, Abhirama Subramanyam Penamakuri
arXiv:2511. 03217v2 Announce Type: replace-cross Abstract: Large language models (LLMs) excel in generating fluent utterances but can lack reliable grounding in verified information.
By Shaghayegh Kolli, Richard Rosenbaum, Timo Cavelius, Lasse Strothe, Andrii Lata, Jana Diesner
E-CONAN introduces Arabic textual entailment and natural inference benchmarks comprising two datasets: E-CONAN-2 (2-way RTE) and E-CONAN-3 (3-way NLI). The datasets are built from automatically-translated pairs, human-validated machine translations, hand-crafted pairs from Arabic teaching books, and rumor-containing news headlines. The authors evaluated nine multilingual pretrained models and five large language models on these benchmarks, demonstrating that E-CONAN offers a more diverse and robust assessment than existing datasets like XNLI and ArNLI.
By Khloud AL Jallad, Nada Ghneim, Ghaida Rebdawi
The paper introduces a new perspective on entity rarity in multimodal entity linking by using knowledge‑graph structural metrics instead of popularity metrics, revealing many rare entities previously overlooked. Experiments show that state‑of‑the‑art models suffer a 15.4–39.9% accuracy drop on these rare‑entity slices. The authors propose a training‑free framework that combines reasoning and retrieval with a vision‑language model, achieving a 6.9% overall accuracy gain and up to 23.3% improvement on rare entities, and release a new benchmark MERLIN‑Rare for focused evaluation.
By Parinthapat Pengpun, Simran Khanuja, Graham Neubig