arXiv AI

Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs

arXiv:2503. 12999v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) have demonstrated exceptional performance in various multi-modal tasks.

arXiv AI

Jun 6

UNIVID: Unified Vision-Language Model for Video Moderation

arXiv:2606. 05748v1 Announce Type: cross Abstract: Global-scale video moderation faces a dual challenge: the need for fine-grained multi-modal reasoning and the demand for interpretable outputs to support downstream enforcement.

By Kejuan Yang, Yizhuo Zhang, Mingyuan Du, Yue Zhang, Dixin Zheng, Kaili Zhao, Yang Xiao, Hanzhong Liang, Kenan Xiao

llms multimodal safety

arXiv Machine Learning

Jun 30

DataComp-VLM: Improved Open Datasets for Vision-Language Models

arXiv:2606. 28551v1 Announce Type: cross Abstract: Building performant Vision-Language Models (VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies.

By Matteo Farina, Vishaal Udandarao, Thao Nguyen, Selim Kuzucu, Maximilian B\"other, Andreas Hochlehnert, Adhiraj Ghosh, Marianna Nezhurina, Karsten Roth, Joschka Struber, Yuhui Zhang, Sebastian Dziadzio, Elaine Sui, Soumya Jahagirdar, Dhruba Ghosh, Hasan Hammoud, Thomas De Min, Simone Caldarella, Jehanzeb Mirza, Sedrick Keh, Mehdi Cherti, Hilde Kuehne, Bernt Schiele, Serena Yeung-Levy, Muhammad Ferjad Naeem, Federico Tombari, Ana Klimovic, Elisa Ricci, Matthias Bethge, Sewoong Oh, Ameya Prabhu, Alessio Tonioni, Jenia Jitsev, Massimiliano Mancini, Ludwig Schmidt, Nikhil Parthasarathy

llms multimodal benchmarks

arXiv AI

Jul 28

DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes

arXiv:2607. 24516v1 Announce Type: cross Abstract: While data curation for Vision Language Models (VLMs) is increasingly active, public practice for constructing pretraining mixtures remains largely heuristic: practitioners stack datasets that pass quality filters, set cross-domain ratios by intuition, and lack a principled, attributable criterion for admitting new data, while frontier recipes remain undisclosed.

By Jiahao Xie, Zhongbin Guo, Qianle Wang, Ruiqi Lu, Dongling Xiao, Wanxuan Sun, Cheng Yang

llms multimodal

arXiv Machine Learning

Jul 20

MLLM-DataEngine: Closing the Loop of Multimodal Instruction Tuning Data Generation

arXiv:2607. 15299v1 Announce Type: cross Abstract: In this paper, we propose MLLM-DataEngine, a novel closed-loop system that bridges data generation, model training, and evaluation.

By Zhiyuan Zhao, Bin Wang, Linke Ouyang, Yiqi Lin, Pan Zhang, Xiaoyi Dong, Jiaqi Wang, Conghui He

llms fine-tuning multimodal benchmarks

arXiv AI

Jun 17

Mordal: Automated Pretrained Model Selection for Vision Language Models

arXiv:2502. 00241v2 Announce Type: replace-cross Abstract: Incorporating multiple modalities into large language models (LLMs) is a powerful way to enhance their understanding of non-textual data, enabling them to perform multimodal tasks.

By Shiqi He, Insu Jang, Mosharaf Chowdhury

llms robotics multimodal benchmarks

arXiv Machine Learning

Jun 18

Semantic Robustness Certification for Vision-Language Models

arXiv:2606. 18839v1 Announce Type: new Abstract: Vision-language models (VLMs) are now widely used in downstream tasks.

By Peiyu Yang, Paul Montague, Feng Liu, Andrew C. Cullen, Amardeep Kaur, Christopher Leckie, Sarah M. Erfani

llms multimodal

arXiv AI

Jun 9

An Effective Router for Vision-Language Model Selection

arXiv:2606. 08970v1 Announce Type: new Abstract: Vision-language models (VLMs) with varying performance and resource requirements are widely deployed, making it difficult for users to select the most appropriate one among numerous VLM candidates.

By Can Wang, Shengwei Wang, Bolin Zhang, Zhiying Tu, Dianhui Chu

llms multimodal

arXiv AI

Jun 19

The Hidden Evolution of Disguised Visual Context inside the VLM

arXiv:2606. 20077v1 Announce Type: cross Abstract: Visual tokens enter Large Language Models (LLMs) as raw, foreign signals.

By Wish Suharitdamrong, Tony Alex, Muhammad Awais, Sara Atito

llms multimodal benchmarks

arXiv AI

Aug 10

SABRE: Scalable and Automated Benchmarking of VLMs under Stress

arXiv:2608. 07435v1 Announce Type: cross Abstract: Vision-language models (VLMs) are improving rapidly, but benchmark development lags behind, making weaknesses hard to identify.

By Zixuan Lan, Luzhe Sun, Matthew R. Walter, Jiawei Zhou

llms multimodal benchmarks

arXiv AI

1d ago

OODBench: Out-of-Distribution Benchmark for Large Vision-Language Models

arXiv:2602. 18094v2 Announce Type: replace-cross Abstract: Existing Visual-Language Models (VLMs) have achieved significant progress by being trained on massive-scale datasets, typically under the assumption that data are independent and identically distributed (IID).

By Ling Lin, Yang Bai, Heng Su, Congcong Zhu, Yaoxing Wang, Yang Zhou, Huazhu Fu, Jingrun Chen

llms agents robotics multimodal benchmarks safety

arXiv AI

Jul 14

DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs

arXiv:2607. 11228v1 Announce Type: cross Abstract: While Large Vision-Language Models (LVLMs) demonstrate remarkable capabilities, they remain highly susceptible to embedded social biases.

By Anqi Li, Jie Zhang, Zhongqi Wang, Songkai Xue, Jiahao Wang, Shiguang Shan, Xilin Chen

llms agents reinforcement-learning multimodal benchmarks safety

Hugging Face Trending Papers

Jul 13

DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs

While Large Vision-Language Models (LVLMs) demonstrate remarkable capabilities, they remain highly susceptible to embedded social biases. Existing bias evaluation protocols predominantly rely on static datasets, which provide only a superficial assessment, as their fixed test cases cannot adaptively evolve to measure the true depth and limits of model vulnerabilities.

llms agents reinforcement-learning multimodal benchmarks safety

Related stories