MiD-VLM is a multimodal model that detects and grounds global semantic misinformation in image-caption pairs. It combines image-text representations from SigLIP with visual scene features from DINOv2, and adapts Qwen2‑VL‑7B using LoRA to jointly predict authenticity, identify manipulation type, and localize misleading caption tokens. On the DGM4+ validation split, it achieves an F1 score of 0.9679 for binary authenticity classification.
By Tarunya Prasad, Bharatesh Chakravarthi
The paper investigates how different multimodal design choices affect the performance of misinformation detection systems. Using over 3,375 experiments across three benchmark datasets and various pre‑trained vision and language models, the authors systematically compare design options and conduct robustness analyses. The study offers practical guidance on which choices improve detection, when they may fail silently, and which pipeline components most influence model behavior, addressing four key research questions.
By Akshit Sharma, Prashant W. Patil
arXiv:2606. 24112v1 Announce Type: new Abstract: Multimodal misinformation detection is increasingly important because viral posts now combine long multilingual narratives, several images, mixed provenance, and subtle text--image framing errors.
By Chenhao Dang, Dantong Zhu, Jun Yang, Conghui He, Weijia Li
arXiv:2601. 14954v3 Announce Type: replace Abstract: Social media increasingly disseminates information through mixed image text posts, but rumors often exploit subtle inconsistencies and forged content, making detection based solely on post content difficult.
By Han Li, Hua Sun
arXiv:2608. 09122v1 Announce Type: cross Abstract: The localized depiction of perceptual quality has long been a crucial, yet underexplored, challenge in image quality assessment (IQA).
By Ziheng Jia, Yingji Liang, Jiaying Qian, Xiongkuo Min
arXiv:2606. 03066v1 Announce Type: new Abstract: The rapid rise of generative AI has made multimodal fake news increasingly realistic and pervasive, posing severe threats to public trust and social stability.
By Jinjie Shen, Yaxiong Wang, Yujiao Wu, Lechao Cheng, Tianrui Hui, Nan Pu, Zhihui Li, Zhun Zhong
Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models introduces the VIG‑Sampler, a method that prioritizes tokens for decoding based on their attention to image tokens and penalizes redundancy in image‑attention distributions. The approach aims to improve the quality of multimodal generation by selecting more informative tokens during diffusion decoding. Experiments on seven captioning and VQA benchmarks with three open‑source dMLLMs show that VIG‑Sampler outperforms the Info‑Gain Sampler by an average of 19.3 CIDEr points and achieves better COCO Caption results using only half as many decoding steps.
By Insu Lee, Wooje Park, Wonseok Shin, Jinwoo Son, Byonghyo Shim
The paper investigates how evaluation protocols and ground‑truth quality affect the classification performance of Multimodal Large Language Models (MLLMs). It identifies and corrects key issues such as discarded out‑of‑list outputs, inflated distractor choices, and poor open‑world mapping, and shows that design choices like batch size, image ordering, and text encoder selection significantly influence accuracy. Using a multilabel reannotation of 625 ImageNet‑1k classes (ReGT), the study finds that corrected labels can boost MLLM performance by up to 10.8%, narrowing the gap with supervised models, and demonstrates that MLLMs can assist human annotators in about half of difficult cases.
By Nikita Kisel, Illia Volkov, Klara Janouskova, Jiri Matas
TokenSwap introduces a method to benchmark and reduce the modality gap in multimodal large language models (MLLMs) by interleaving semantically aligned images with text tokens. By transforming text-only benchmarks like MMLU into image‑interleaved versions (TokenSwap‑Bench), the authors reveal a pervasive performance drop—averaging 19.6%—across 42 MLLMs, with reasoning models showing smaller gaps. Training with TokenSwap mitigates this gap while maintaining strong text‑only and vision‑language performance.
By Andong Hua, Colton Bishop, Igor Mordatch, Arian Hosseini, Jindong Gu, Aleksandra Faust, Rebecca Roelofs, Yao Qin
arXiv:2506. 03096v2 Announce Type: replace-cross Abstract: Contrastive language-image pre-training aligns features of text-image pairs in a common latent space via distinct encoders for each modality.
By Christian Schlarmann, Francesco Croce, Nicolas Flammarion, Matthias Hein
arXiv:2606. 04205v1 Announce Type: cross Abstract: The growing popularity and capacity of generative models have eroded the distinction between human and machine-generated content, motivating a growing body of work on detection across text, images, and audio.
By Sajad Ebrahimi, Nima Jamali, Bardia Shirsalimian, Kelly McConvey, Wentao Zhang, Jalehsadat Mahdavimoghaddam, Maksym Taranukhin, Maura Grossman, Vered Shwartz, Yuntian Deng, Ebrahim Bagheri
arXiv:2609.26907v1 Announce Type: cross
Abstract: Memes often derive their harmful, hateful, or sarcastic meaning from small but decisive visual, textual, or cross-modal cues. Existing multimodal cla...
By Akshit Sharma, Prashant W. Patil