arXiv Computation and Language
3d ago

MMDS-Bench: Benchmarking Multimodal Large Language Models on Dynamic Stance in Social Media Interactions

MMDS-Bench is a new diagnostic benchmark for multimodal dynamic stance classification in social media parent‑reply interactions. It contains 3,482 multimodal instances annotated with a seven‑label stance taxonomy, plus an 800‑instance subset that demands structured reasoning over parent and reply understanding and stance‑relation inference. The benchmark also tags each instance with five challenge factors—multimodal fusion, parent framing, non‑literal expression, interaction reasoning, and label‑boundary ambiguity—and evaluates 12 multimodal large language models using a reference‑grounded LLM‑judge protocol, revealing that current models still struggle with relational inference beyond separate parent and reply comprehension.

By Yuzhe Ding, Kang He, Li Zheng, Shengwu Zheng, Teng Shi, Fei Li, Chong Teng, Donghong Ji