arXiv AI By Ritabrata Chakraborty, Rajatsubhra Chakraborty, Shivakumara Palaiahnakote, Angelo Cangelosi, Umapada Pal

Vision-Language Models are Fragile Multilingual Associators

Read the original on arXiv AI →

arXiv:2608. 12333v1 Announce Type: cross Abstract: Vision-language models must associate visual entities with textual attributes.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 16

Same Answer, Different Representations: Hidden instability in VLMs

arXiv:2602.06652v2 Announce Type: replace Abstract: The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions r...

By Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena, Aryo Pradipta Gema, Wai-Chung Kwan, Fazl Barez, Maria Sofia Bucarelli, Fabrizio Silvestri, Pasquale Minervini
arXiv Computation and Language
Aug 28

Not Truly Multilingual: Script Consistency as a Missing Dimension in VLM Evaluation

The paper introduces PuMVR, a benchmark of 1,000 Punjabi image‑text pairs spanning three scripts—Gurmukhi, Shahmukhi, and Roman—to evaluate Vision‑Language Models (VLMs). Testing ten state‑of‑the‑art VLMs reveals a significant Script Gap: models perform well in one script but poorly in another, with accuracy differences up to 16%. The authors propose the Script Consistency Rate (SCR) as a new metric, noting it can be as low as 24.8% on their benchmark, and argue that current multilingual VLMs are not truly multi‑script.

By Prabhjot Singh, Bhushan Pawar, Madhu Reddiboina, Rajvee Sheth