Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

4,615 stories · RSS feed

arXiv AI

Jun 16

Dual-Uncertainty Guided Policy Learning for Multimodal Reasoning

arXiv:2510. 01444v3 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards (RLVR) has advanced reasoning capabilities in multimodal large language models.

By Rui Liu, Dian Yu, Tong Zheng, Runpeng Dai, Zongxia Li, Wenhao Yu, Zhenwen Liang, Linfeng Song, Haitao Mi, Pratap Tokekar, Dong Yu

llms reinforcement-learning multimodal benchmarks

arXiv AI

Jun 16

Binary Tracking for Spatial QA and Navigation with Open Vision-Language Models

arXiv:2606. 16902v1 Announce Type: cross Abstract: This work addresses spatial question answering for service robots traversing long egocentric routes.

By Dongbin Na, Chanwoo Kim, Soonbin Rho, Giyun Choi, Gangbok Lee, Dooyoung Hong

llms rag agents nlp robotics multimodal benchmarks

arXiv AI

Jun 16

Decoupling Semantics from Distortions: Multi-Scale Two-Stream Vision-Language Alignment for AI-Generated Image Quality Assessment

arXiv:2606. 16799v1 Announce Type: cross Abstract: Existing vision-language model (VLM)-based AI-generated image quality assessment (AIGIQA) methods suffer from a fundamental semantic-distortion dimensional conflict: monolithic representations optimized for semantic discrimination inherently entangle compositional understanding with low-level perceptual sensitivity, rendering them blind to fine-grained quality degradations.

By Zijie Meng

llms efficiency multimodal benchmarks safety

arXiv AI

Jun 16

XAI-Grounded Explanation Generation for Speech Deepfake Detection with Training-Free Multimodal Large Language Models

arXiv:2606. 16137v1 Announce Type: cross Abstract: Speech deepfake detection (SDD) systems require trustworthy explanations for reliable decision-making.

By Yupei Li, Qiyang Sun, Xiaoliang Wu, Chenxi Wang, Berrak Sisman, Bj\"orn W. Schuller

llms multimodal

arXiv AI

Jun 16

Unified Multimodal Model for Brain MRI Imputation and Understanding

arXiv:2606. 16484v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) hold great potential for medicine, as they inherit knowledge from LLM and allow multiple data modalities to be integrated, analysed and interpreted in natural language.

By Zhiyun Song, Che Liu, Tian Xia, Avinash Kori, Wenjia Bai

llms rag multimodal safety

arXiv AI

Jun 16

FlowMPC: Improving Flow Matching policies with World Models

arXiv:2606. 16286v1 Announce Type: cross Abstract: Flow Matching (FM) is a powerful approach for behavior cloning in multimodal action spaces [Jiang et al.

By Chandon Hamel

diffusion robotics multimodal

arXiv AI

Jun 16

Learned Image Compression for Vision-Language-Action Models

arXiv:2606. 16253v1 Announce Type: cross Abstract: Vision-language-action (VLA) models increasingly rely on high-frequency multi-camera observations, making visual communication a major bottleneck for real-time robotic control in bandwidth-constrained or distributed deployment settings.

By Hyeonjun Kim, Jegwang Ryu, Sangbeom Ha, Junhyeok Lee, Jun-Hyuk Kim, Hyemin Ahn, Jaeho Lee

robotics multimodal benchmarks

arXiv AI

Jun 16

Tool-IQA: Augmenting Image Quality Assessment with Simple Tools

arXiv:2606. 16082v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have been increasingly adopted for Image Quality Assessment (IQA).

By Guanyi Qin, Junjie Zhang, Chunming He, Yibing Fu, Jie Liang, Tianhe Wu, Lei Zhang

llms agents multimodal benchmarks

arXiv AI

Jun 16

MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA

arXiv:2606. 15906v1 Announce Type: cross Abstract: Long-document multimodal question answering requires a system to locate sparse evidence in long PDFs and integrate clues from text, tables, images, charts, and complex layouts.

By Yilong Zuo, Xunkai Li, Jing Yuan, Qiangqiang Dai, Hongchao Qin, Ronghua Li

rag agents nlp multimodal

arXiv AI

Jun 16

Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models

arXiv:2606. 15880v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have been increasingly adopted in forensics for their robust semantic understanding.

By Kaiqing Lin, Zhiyuan Yan, Ruoxin Chen, Ke-Yue Zhang, Yue Zhou, Caiyong Piao, Bin Li, Taiping Yao, Bo Wang, Youchang Xiao, Shouhong Ding

llms fine-tuning multimodal benchmarks

arXiv AI

Jun 16

LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies

arXiv:2606. 15768v1 Announce Type: cross Abstract: Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene.

By Jialei Chen, Kai Wang, Kang Chen, Shuaihang Chen, Feng Gao, Wenhao Tang, Zhiyuan Li, Weilin Liu, Zhuyu Yao, Boxun Li, Yuanbo Xu, Chao Yu

diffusion robotics multimodal benchmarks

arXiv AI

Jun 16

MAF: Multimodal Adaptive Few-shot Prompting for Sentiment Analysis with MLLMs

arXiv:2606. 15694v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in understanding complex multimodal content.

By Hangling Xie

llms nlp multimodal benchmarks

arXiv AI

Jun 16

Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment

arXiv:2606. 15441v1 Announce Type: cross Abstract: Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution.

By Lipeng He, Yihan Wang, Jiawen Zhang, N. Asokan

llms rag agents reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 16

Post-Launch Capability Expansion of Vision-Language Models via Prompting for On-Orbit Spacecraft Inspection

arXiv:2606. 15427v1 Announce Type: cross Abstract: Spaceborne inspection systems often deploy perception models prior to launch, after which updating model weights or expanding fixed label sets becomes operationally impractical.

By Nicholas A. Welsh, Lennon J. Shikhman, Monty Nehru Attazs, Seemanthini K. Putane, Van Minh Nguyen, Ryan T. White

llms computer-vision multimodal

arXiv AI

Jun 16

CAP: Towards PPG Universal Representation Learning with Patient-level Supervision

arXiv:2606. 15284v1 Announce Type: cross Abstract: Photoplethysmography (PPG) plays a central role in wearable health monitoring and clinical decision support.

By Chenyang He, Xinyi Shao, Shun Huang, Bosong Huang, Daoqiang Zhang, Ming Jing, Cheng Ding

multimodal benchmarks safety

arXiv AI

Jun 16

Driving, Fast or Slow? Neuro-Symbolic Guidance for Motion Prediction in Multi-Modal Ground Mobility

arXiv:2606. 15251v1 Announce Type: cross Abstract: Accurate and interpretable motion prediction for heterogeneous traffic spaces, including pedestrians, bicycles, cars, and trucks, is essential for safe autonomous navigation.

By Simon Kohaut, Felix Divo, Julius Hahnewald, Benedict Flade, Julian Eggert, Kristian Kersting, Devendra Singh Dhami

agents multimodal benchmarks

arXiv AI

Jun 16

MapDream: Task-Driven Map Learning for Vision-Language Navigation

arXiv:2602. 00222v3 Announce Type: replace-cross Abstract: Vision-Language Navigation (VLN) requires agents to follow natural language instructions in partially observed 3D environments, motivating map representations that aggregate spatial context beyond local perception.

By Guoxin Lian, Shuo Wang, Yucheng Wang, Yongcai Wang, Maiyue Chen, Kaihui Wang, Bo Zhang, Zhizhong Su, Deying Li, Zhaoxin Fan

agents fine-tuning multimodal benchmarks

arXiv AI

Jun 16

TimeVista: Exploring and Exploiting Vision-Language Models as Judges for Time Series Forecasting

arXiv:2606. 16173v1 Announce Type: new Abstract: High-quality time series forecasting is pivotal for real-world decision-making.

By Zhi Chen, Yuxuan Wang, Jialong Wu, Yong Liu, Haoran Zhang, Xingjian Su, Jianmin Wang, Mingsheng Long

llms multimodal benchmarks

arXiv AI

Jun 16

Teacher-Student Structure for Domain Adaptation in Ensemble Audio-Visual Video Deepfake Detection

arXiv:2606. 15117v1 Announce Type: cross Abstract: The rapid advancement of generative AI models is leading to more realistic deepfake media, encompassing the manipulation of audio, video, or both.

By Elham Abolhasani, Maryam Ramezani, Hamid R. Rabiee

robotics fine-tuning multimodal safety

arXiv AI

Jun 16

Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretability

arXiv:2606. 14758v1 Announce Type: cross Abstract: As Vision-Language Models are increasingly deployed in safety-critical applications, the trustworthiness of their explanations becomes crucial.

By Emirhan Bilgi\c{c}, Baptiste Caramiaux, Zhi Yan, Gianni Franchi

llms rag multimodal safety

Get the daily briefing