Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

8,506 stories · RSS feed

arXiv AI

Sep 1

LeMat-Synth: a multi-modal toolbox to curate broad synthesis procedure databases from scientific literature

arXiv:2510.26824v2 Announce Type: replace-cross Abstract: Wide access to advanced experimental methods in materials science has given rise to an abundance of procedural knowledge, which is scattered...

By Magdalena Lederbauer, Siddharth Betala, Valerie Gentzke, Anamaria Leonescu, Amine Sehaba, Faris Flaifil, Ayush Jain, Alfonso Amayuelas, Nikhil Yelamarthy, Xiyao Li, Gr\'egoire Germain, Stefano Ribes, Stefan P. Schmid, Alexandre Nozadze, Anna Kelmanson, Sudheesh Kumar Ethirajan, Mohd Zaki, Elton Pan, Georgia Channing, Connor W. Coley, Philippe Schwaller, Roc\'io Mercado, Alexandre Duval, Mathilde L. D. Franckel, Samuel P. Gleason

llms multimodal benchmarks

arXiv AI

Sep 1

Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

arXiv:2603.10807v2 Announce Type: replace-cross Abstract: Existing LLM safety evaluations rely on binary attack-success rates and domain-agnostic taxonomies, leaving regulated Banking, Financial Serv...

By Fabrizio Dimino, Bhaskarjit Sarmah, Stefano Pasquali

llms multimodal benchmarks safety

arXiv AI

Sep 1

SketchVLM: Vision language models can annotate images to explain thoughts and guide users

arXiv:2604.22875v3 Announce Type: replace-cross Abstract: When answering questions about images, humans naturally point, label, and draw to explain their reasoning. In contrast, modern vision-languag...

By Brandon Collins, Logan Bolton, Hung Huy Nguyen, Mohammad Reza Taesiri, Trung Bui, Anh Totti Nguyen

llms fine-tuning multimodal benchmarks

arXiv AI

Sep 1

Vision-Language Models Suppress Female Representations Under Ambiguous Input

arXiv:2605.31556v2 Announce Type: replace-cross Abstract: Alignment teaches vision-language models (VLMs) to avoid expressing demographic biases, and when gender is clearly visible they largely succe...

By Arnau Marin-Llobet, Simon Henniger, Mahzarin R. Banaji

llms rag multimodal benchmarks safety

arXiv AI

Sep 1

FSA-GRPO: Teaching Auditory LLMs to Use Few-Shot Demonstrations

arXiv:2606.02615v2 Announce Type: replace-cross Abstract: Few-shot prompting provides an effective way to adapt auditory large language models to low-resource tasks such as children's speech recognit...

By Haolong Zheng, Siyin Wang, Xulin Fan, Zengrui Jin, Mark Hasegawa-Johnson

llms reinforcement-learning multimodal

arXiv Machine Learning

Sep 1

Language-Informed Flow Matching for Trend-Guided Structure-Based 3D Molecular Generation

arXiv:2608.31009v1 Announce Type: new Abstract: Structure-based drug design (SBDD) requires ligands that satisfy both 3D target affinity and 1D chemical validity. Existing controllable generation met...

By Tianyu Gao, Zhikai Su, Jiashu Li, Wenjun Gao, Zichuan Ying, Zhe Zhao, Fei Zhang, Ye Wei

agents diffusion fine-tuning multimodal

arXiv Machine Learning

Sep 1

VisER: Visual Evidence and Reliance for Object Hallucination Detection in LVLMs

arXiv:2608.30480v1 Announce Type: cross Abstract: Object hallucination remains a persistent reliability issue in large vision-language models, where generated object mentions may sound plausible but...

By Afsaneh Hasanebrahimi, Hanxun Huang, Christopher Leckie, Sarah Erfani

llms multimodal benchmarks safety

arXiv Machine Learning

Sep 1

Diverse via bounded Agreement: Geometric Regularization for Multimodal Fusion

arXiv:2601.21670v4 Announce Type: replace-cross Abstract: Multimodal fusion is often treated as an optimization-balancing problem, where training signals are adjusted to prevent one modality from dom...

By Zixuan Xia, Hao Wang, Pengcheng Weng, Yanyu Qian, Yangxin Xu, William Dan, Fei Wang

rag multimodal benchmarks safety