arXiv Computation and Language

M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset

arXiv Computation and Language
Sep 11

Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

The paper introduces a new perspective on entity rarity in multimodal entity linking by using knowledge‑graph structural metrics instead of popularity metrics, revealing many rare entities previously overlooked. Experiments show that state‑of‑the‑art models suffer a 15.4–39.9% accuracy drop on these rare‑entity slices. The authors propose a training‑free framework that combines reasoning and retrieval with a vision‑language model, achieving a 6.9% overall accuracy gain and up to 23.3% improvement on rare entities, and release a new benchmark MERLIN‑Rare for focused evaluation.

By Parinthapat Pengpun, Simran Khanuja, Graham Neubig
arXiv AI
Sep 1

SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models

SafeAtlas-VL introduces a large multimodal safety dataset with 1.5 million instances, rating image, request, and response risks on a five‑level ordinal scale across 15 harm categories and 55 subcategories. The accompanying SafeAtlas‑Bench provides 5,000 held‑out cases for evaluating ordinal predictions and continuous risk scores. Models trained on this data, including an 8B Guard model, achieve state‑of‑the‑art performance, outperforming prior benchmarks by about 4% in F1 score.

By Zongrui Wang, Xiangyang Zhu, Sicheng Wang, Han Wang, Dingyi Rong, Zeyu Zhang, Chunyi Li, Yue Shi, Kaiwei Zhang, Zicheng Zhang, Yuan Tian, Qi Jia, Yan Teng, Wei Sun, Ning Liu, Guangtao Zhai
arXiv AI
Jun 10

MMClima: A Framework for Multimodal Climate Science Data and Evaluation

arXiv:2606. 10194v1 Announce Type: cross Abstract: Climate change research increasingly requires AI systems that reason across text, dynamic visual content, and scientific figures, yet existing climate QA benchmarks are small, mostly textual, and cover a narrow range of models.

By Muhammad Umer Sheikh, Hassan Abid, Khawar Shehzad, Ufaq Khan, Muhammad Haris Khan