Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

4,615 stories · RSS feed

arXiv Machine Learning

Jun 16

DiRecT: Safe Diffusion-Based Planning via Receding-Horizon Denoising

arXiv:2606. 15359v1 Announce Type: new Abstract: Diffusion models have emerged as powerful tools for planning and control by learning multimodal distributions over actions and trajectories.

By Paolo Giaretta, Zeyang Li, Navid Azizan

diffusion multimodal benchmarks safety

arXiv AI

Jun 16

InstantForget: Update-Free Backdoor Unlearning with Inference-Time Feature Reset

arXiv:2606. 15730v1 Announce Type: cross Abstract: Backdoor unlearning aims to remove a malicious trigger behavior from a deployed model while preserving clean utility.

By Zhenyu Yu

multimodal

arXiv AI

Jun 16

Agentic Framework for Deep Learning workload migration via In-Context Learning

arXiv:2606. 15994v1 Announce Type: new Abstract: Translating deep learning models from PyTorch's flexible, object-oriented design to JAX's functional, stateless setup is usually a manual and error-prone task.

By Qiyue Liang, Steven Ingram, George Vanica, Andi Gavrilescu, Newfel Harrat, Hassan Sipra, Sethuraman Sankaran

llms agents multimodal benchmarks safety

arXiv AI

Jun 16

UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analytics

arXiv:2606. 15890v1 Announce Type: new Abstract: Understanding urban wellbeing from multimodal data requires integrating heterogeneous spatial and temporal signals, posing significant challenges for current multimodal large language models (MLLMs).

By Yanxin Xi, Xiang Su, Jie Feng, Yu Liu, Sasu Tarkoma, Pan Hui

llms multimodal benchmarks safety

arXiv AI

Jun 16

Seeing Roads Through Words: A Language-Guided Framework for RGB-T Driving Scene Segmentation

arXiv:2602. 07343v2 Announce Type: replace-cross Abstract: Robust semantic segmentation of road scenes under adverse illumination, lighting, and shadow conditions remain a core challenge for autonomous driving applications.

By Ruturaj Reddy, Hrishav Bakul Barua, Junn Yong Loo, Thanh Thi Nguyen, Ganesh Krishnasamy

llms rag agents computer-vision robotics multimodal benchmarks

arXiv Machine Learning

Jun 16

Unsupervised Learning for Missing Modalities in Multimodal Learning

arXiv:2606. 15743v1 Announce Type: new Abstract: This paper addresses the missing-modality challenge in multi-modal learning by introducing Unsupervised Learning for Missing Modalities in Multi-Modal Learning (UL4M4), a flexible framework that imputes missing feature embeddings in a task-independent manner before supervised prediction.

By Hassan Ismkhan, Hamid Bouchahcia

rag multimodal benchmarks

arXiv AI

Jun 16

Teacher-Student Structure for Domain Adaptation in Ensemble Audio-Visual Video Deepfake Detection

arXiv:2606. 15117v1 Announce Type: cross Abstract: The rapid advancement of generative AI models is leading to more realistic deepfake media, encompassing the manipulation of audio, video, or both.

By Elham Abolhasani, Maryam Ramezani, Hamid R. Rabiee

robotics fine-tuning multimodal safety

arXiv AI

Jun 16

Driving, Fast or Slow? Neuro-Symbolic Guidance for Motion Prediction in Multi-Modal Ground Mobility

arXiv:2606. 15251v1 Announce Type: cross Abstract: Accurate and interpretable motion prediction for heterogeneous traffic spaces, including pedestrians, bicycles, cars, and trucks, is essential for safe autonomous navigation.

By Simon Kohaut, Felix Divo, Julius Hahnewald, Benedict Flade, Julian Eggert, Kristian Kersting, Devendra Singh Dhami

agents multimodal benchmarks

arXiv AI

Jun 16

LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies

arXiv:2606. 15768v1 Announce Type: cross Abstract: Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene.

By Jialei Chen, Kai Wang, Kang Chen, Shuaihang Chen, Feng Gao, Wenhao Tang, Zhiyuan Li, Weilin Liu, Zhuyu Yao, Boxun Li, Yuanbo Xu, Chao Yu

diffusion robotics multimodal benchmarks

arXiv Machine Learning

Jun 16

A Multi-Center Benchmark for Abdominal Disease Diagnosis and Report Generation from Non-Contrast CT

arXiv:2606. 16991v1 Announce Type: cross Abstract: Multiphasic contrast-enhanced CT (CECT) is widely used for abdominal lesion characterization, yet it carries inherent risks of contrast-induced nephropathy, escalates acquisition burden, and heavily contributes to radiologist workload.

By Mariam Elbakry, Aliaa Sayed Sheha, Salma Hassan Tantawy, Aya Yassin, Concetto Spampinato, Karim Lekadir, Xiaomeng Li, Marawan Elbatel

multimodal benchmarks

arXiv AI

Jun 16

Learned Image Compression for Vision-Language-Action Models

arXiv:2606. 16253v1 Announce Type: cross Abstract: Vision-language-action (VLA) models increasingly rely on high-frequency multi-camera observations, making visual communication a major bottleneck for real-time robotic control in bandwidth-constrained or distributed deployment settings.

By Hyeonjun Kim, Jegwang Ryu, Sangbeom Ha, Junhyeok Lee, Jun-Hyuk Kim, Hyemin Ahn, Jaeho Lee

robotics multimodal benchmarks

arXiv AI

Jun 16

Dual-Uncertainty Guided Policy Learning for Multimodal Reasoning

arXiv:2510. 01444v3 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards (RLVR) has advanced reasoning capabilities in multimodal large language models.

By Rui Liu, Dian Yu, Tong Zheng, Runpeng Dai, Zongxia Li, Wenhao Yu, Zhenwen Liang, Linfeng Song, Haitao Mi, Pratap Tokekar, Dong Yu

llms reinforcement-learning multimodal benchmarks

arXiv AI

Jun 16

Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification

arXiv:2603. 24058v2 Announce Type: replace-cross Abstract: Object hallucination in Large Vision-Language Models (LVLMs) severely compromises their reliability in real-world applications, posing a critical barrier to their deployment in high-stakes scenarios such as autonomous driving and medical image analysis.

By Han Sun, Qin Li, Peixin Wang, Min Zhang

llms agents robotics multimodal benchmarks safety

arXiv Machine Learning

Jun 16

SDVDiag: Multimodal Causal Discovery for Online Diagnosis in Software-defined Vehicles

arXiv:2606. 15559v1 Announce Type: cross Abstract: The transition toward software-defined vehicles concentrates an increasing share of vehicle functionality into distributed software services, where failures propagate through service dependencies and the surface symptom is often several causal hops away from the underlying defect.

By Matthias Wei{\ss}, Athreya Hosahalli Prakash, Falk Dettinger, Nasser Jazdi, Michael Weyrich

rag agents multimodal

arXiv Machine Learning

Jun 16

A Multimodal Approach to Alzheimer's Diagnosis: Geometric Insights from Cube Copying and Cognitive Assessments

arXiv:2512. 16184v2 Announce Type: replace Abstract: Early and accessible detection of Alzheimer's disease (AD) remains a critical clinical challenge, and cube-copying tasks offer a simple yet informative assessment of visuospatial function.

By Jaeho Yang, Kijung Yoon

multimodal safety

arXiv AI

Jun 16

Efficient Reinforcement for Visual-Textual Thinking with Discrete Diffusion Model

arXiv:2606. 14792v1 Announce Type: cross Abstract: RL-based post-training has been widely adopted to enable interleaved visual and textual reasoning in unified multimodal models capable of both text and image generation.

By Yoonjeon Kim, Yuhta Takida, Chieh-Hsin Lai, Eunho Yang, Yuki Mitsufuji

diffusion reinforcement-learning multimodal

arXiv AI

Jun 16

Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretability

arXiv:2606. 14758v1 Announce Type: cross Abstract: As Vision-Language Models are increasingly deployed in safety-critical applications, the trustworthiness of their explanations becomes crucial.

By Emirhan Bilgi\c{c}, Baptiste Caramiaux, Zhi Yan, Gianni Franchi

llms rag multimodal safety

arXiv AI

Jun 16

Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference

arXiv:2606. 15782v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have demonstrated strong capabilities in vision-language understanding and natural-language response generation.

By Pratheswaran Hariharan, Haiping Xu, Donghui Yan

llms rag multimodal safety

arXiv AI

Jun 16

Is My Vision-Language Data in Your AI? Membership Inference Test (MINT) Demo 2

arXiv:2606. 14748v1 Announce Type: cross Abstract: We present the Membership Inference Test (MINT) Demo 2, a framework designed to improve transparency in machine learning training processes.

By Daniel DeAlcala, Gonzalo Mancera, Julian Fierrez, Aythami Morales, Ruben Tolosana, Ruben Vera-Rodriguez

llms multimodal benchmarks

arXiv AI

Jun 16

OSGuard: A Benchmark for Safety in Computer-Use Agents

arXiv:2606. 15034v1 Announce Type: new Abstract: Computer-use agents are increasingly evaluated by whether they complete realistic desktop and web tasks.

By Mina Mohammadmirzaei, Jeffrey Flanigan

agents multimodal benchmarks safety

Get the daily briefing