← Back to all news
arXiv Computer Vision By Akshit Sharma, Prashant W. Patil

MemeTAG: Keyword-Driven Meme Classification through Tag Embedding Reconstruction

Read the original on arXiv Computer Vision →

MemeTAG introduces a dual‑objective framework for classifying harmful memes by combining keyword generation from a pretrained Vision‑Language Model with an Aggregated Tag Inference Network (ATIN) that condenses these keywords into a rich semantic embedding. The embedding is used as a target for an auxiliary reconstruction loss, encouraging deep alignment between visual and textual features. This approach, along with a three‑stage training strategy, achieves new state‑of‑the‑art results on the HarMeme, Hateful Memes Challenge, and PrideMM datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Aug 25

From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignment

The paper introduces MemeMind, a large-scale dataset for detecting harmful memes that includes a detailed taxonomy and Chain-of-Thought reasoning annotations. It also proposes MemeGuard, a multimodal framework that uses a three-stage training strategy to improve visual understanding, reasoning, and discrimination of harmful content. Experiments show MemeGuard surpasses current state-of-the-art methods on MemeMind, advancing detection accuracy and interpretability.

By Hexiang Gu, Qifan Yu, Yuan Liu, Zikang Li, Saihui Hou, Jian Zhao, Zhaofeng He
multimodalbenchmarkssafety
More like this →
arXiv AI
6d ago

MemeLens: Multilingual Multitask VLMs for Memes

MemeLens is a unified multilingual, multitask Vision‑Language Model designed to improve meme understanding across a wide range of tasks such as hate, misogyny, propaganda, sentiment, and humour. The authors consolidated 38 public meme datasets, mapping their labels into a shared taxonomy of 20 tasks covering harm, targets, figurative intent, and affect, and conducted extensive experiments to show that multimodal training and a unified approach outperform fine‑tuning on individual datasets. All experimental resources, the model, and the datasets are released publicly for community use.

By Ali Ezzat Shahroor, Mohamed Bayan Kmainasi, Abul Hasnat, Dimitar Dimitrov, Giovanni Da San Martino, Preslav Nakov, Firoj Alam
llmsroboticsfine-tuningmultimodal
More like this →
arXiv Machine Learning
Jun 2

Reconstructing Content via Collaborative Attention to Improve Multimodal Embedding Quality

arXiv:2603. 01471v2 Announce Type: replace-cross Abstract: Multimodal embedding models, rooted in multimodal large language models (MLLMs), have yielded significant performance improvements across diverse tasks such as retrieval and classification.

By Jiahan Chen, Da Li, Hengran Zhang, Yinqiong Cai, Lixin Su, Jiafeng Guo, Daiting Shi, Dawei Yin, Keping Bi
llmsragmultimodal
More like this →