MultiInpaintCOCO extends the English InpaintCOCO dataset to 13 typologically diverse languages, creating a multilingual benchmark of 953 minimal-pair examples focused on fine-grained visual details. The study evaluates state-of-the-art multilingual multimodal retrieval models, revealing that performance drops significantly for lower-resource languages and that many models exhibit a systematic bias toward English captions. The benchmark and code are publicly available at https://github.com/ufal/multi-inpaint-coco.
By Andrei-Alexandru Manea, Jind\v{r}ich Libovick\'y, Philipp J. R\"osch
arXiv:2605.26380v2 Announce Type: replace-cross
Abstract: Frontier multimodal large language models (MLLMs) have been reported to achieve over 90\% accuracy on fine-grained perception benchmarks. How...
By Jingru Chen, Yiming Liu, Mingtao Chen, Sijie Chen, Richeng Xuan, Liang Yang, Zhichao Hu, Fanyang Lu
BanglaVerse is a new benchmark that evaluates multilingual vision‑language models on Bengali culture, covering nine visual domains and expanding to four languages and five Bangla dialects for a total of about 32,200 artifacts. It includes visual question answering and captioning tasks built from 1,152 manually curated images. Experiments show that models perform worse on dialectal variants and that missing cultural knowledge, rather than visual grounding, is the main bottleneck.
By Nurul Labib Sayeedi, Md. Faiyaz Abdullah Sayeedi, Shubhashis Roy Dipta, Mahbub E Sobhani, Rubaya Tabassum, Ariful Ekraj Hridoy, Mehraj Mahmood, Md. Tarek Hasan, Swakkhar Shatabda
arXiv:2510.26861v4 Announce Type: replace-cross
Abstract: Multimodal retrieval systems are expected to operate in a semantic space, agnostic to the language or cultural origin of the query. In practi...
By Teerapol Saengsukhiran, Peerawat Chomphooyod, Narabodee Rodjananant, Chompakorn Chaksangchaichot, Patawee Prakrankamanant, Witthawin Sripheanpol, Pak Lovichit, Sarana Nutanong, Ekapol Chuangsuwanich
arXiv:2604. 18347v2 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) achieved rapid progress in the recent years.
By Daniela Baiamonte, Elena Fano, Matteo Gabburo, Stefano Simonazzi, Leonardo Rigutini, Andrea Zugarini
UniFGVC introduces a training‑free framework for few‑shot fine‑grained visual classification that reframes the task as a multimodal retrieval problem. It uses a Category‑Discriminative Visual Captioner to generate attribute‑aware textual descriptions, then matches image‑description pairs with category templates via pre‑trained encoders. Experiments on 12 datasets show that UniFGVC consistently outperforms existing CLIP‑based and fully‑supervised MLLM methods, demonstrating the effectiveness of the retrieval paradigm.
By Haiyun Guo, Hongyu Guo, Xiangzhao Hao, Jiarui Guo, Jinqiao Wang, Tat-Seng Chua
The paper introduces PuMVR, a benchmark of 1,000 Punjabi image‑text pairs spanning three scripts—Gurmukhi, Shahmukhi, and Roman—to evaluate Vision‑Language Models (VLMs). Testing ten state‑of‑the‑art VLMs reveals a significant Script Gap: models perform well in one script but poorly in another, with accuracy differences up to 16%. The authors propose the Script Consistency Rate (SCR) as a new metric, noting it can be as low as 24.8% on their benchmark, and argue that current multilingual VLMs are not truly multi‑script.
By Prabhjot Singh, Bhushan Pawar, Madhu Reddiboina, Rajvee Sheth
The paper introduces Vision-Free Adaptation (VFA), a method that separates multilingual language enhancement from visual alignment in multimodal large language models. VFA fine‑tunes a base LLM on multilingual text to create a multilingual task vector, which is then merged with the vision‑aligned task vector of an existing MLLM. Experiments on five MLLMs and six multilingual benchmarks show consistent gains while preserving multimodal and text‑only performance, and using less than 2% of text data narrows the performance gap to fully multimodal‑trained models.
By Yixia Li, Yaqing Shi, Zhiwen Ruan, Dongdong Zhang, Lingjie Jiang, Shaohan Huang, Yun Chen, Guanhua Chen, Furu Wei
Pixel language models treat text as rendered images, enabling strong cross‑lingual transfer. This study trains MPIXEL models varying languages, scripts, and capacity while keeping per‑language compute constant, and evaluates them on five tasks across 83 languages. Results show that scaling improves performance on added languages and scripts, mitigates the curse of multilinguality, and outperforms subword and character baselines on typologically distant low‑resource languages, though transfer to unseen scripts does not benefit further from scaling.
By Ilker Kesen, Jenny Kunz, Kasra Malihi, Desmond Elliott
The paper evaluates 14 state‑of‑the‑art text‑to‑video retrieval models on three major datasets using a unified framework. It finds that short, clear, and simple captions—especially those describing single actions or color attributes—yield higher recall, while complex, multi‑step events or fine‑grained scene descriptions remain difficult for all models. Attention‑driven architectures better handle temporally dependent or multi‑step queries, whereas dual‑encoder and multimodal fusion models excel mainly on simpler captions, and cross‑dataset generalization improves with larger, more diverse caption sets but generative captions do not consistently boost accuracy.
By Maria-Eirini Pegia, Dimitrios Stefanopoulos, Bj\"orn {\TH}\'or J\'onsson, Anastasia Moumtzidou, Ilias Gialampoukidis, Stefanos Vrochidis, Ioannis Kompatsiaris
arXiv:2609.06245v1 Announce Type: cross
Abstract: Multimodal Large Language Models (MLLMs) perform strongly on general visual understanding tasks such as visual question answering, yet they often str...
By Yixin Wan, Tianle Zheng, Kai-Wei Chang
arXiv:2506. 08774v2 Announce Type: replace-cross Abstract: Different machine learning models can represent the same underlying concept in different ways.
By Fan Xu, Luis A. Leiva