Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

4,615 stories · RSS feed

arXiv Machine Learning

Jun 16

Robust Neural Tucker Factorization with Bias Correction and Adaptive Initialization

arXiv:2606. 16388v1 Announce Type: new Abstract: High-dimensional incomplete (HDI) tensors are widely used in traffic and climate applications, but sparse observations make accurate completion difficult.

By Yuchao Su, Yixin Ran

rag multimodal safety

arXiv AI

Jun 16

TechRAG: Evidence-Gated Multimodal Agentic RAG for Technical Literature Reasoning

arXiv:2606. 01613v2 Announce Type: replace-cross Abstract: This paper presents an agentic multimodal retrieval-augmented generation (RAG) framework for domain-specific literature reasoning, instantiated on a curated corpus of several thousand papers in intelligent tires, vehicle dynamics, vehicle control, sensing, estimation, and machine learning.

By Kanwar Bharat Singh

llms rag agents multimodal

arXiv AI

Jun 16

Are Neuro-Inspired Multi-Modal Vision-Language Models Resilient to Membership Inference Privacy Leakage?

arXiv:2511. 20710v2 Announce Type: replace-cross Abstract: In the age of agentic AI, the growing deployment of multi-modal models (MMs) has introduced new attack vectors that can leak sensitive training data in MMs, causing privacy leakage.

By David Amebley, Sayanton Dibbo

llms agents multimodal benchmarks safety

arXiv AI

Jun 16

UniT: Unified Multimodal Chain-of-Thought Test-time Scaling

arXiv:2602. 12279v2 Announce Type: replace-cross Abstract: Unified models can handle both multimodal understanding and generation within a single architecture, yet they typically operate in a single pass without iteratively refining their outputs.

By Leon Liangyu Chen, Haoyu Ma, Zhipeng Fan, Ziqi Huang, Animesh Sinha, Xiaoliang Dai, Jialiang Wang, Zecheng He, Jianwei Yang, Chunyuan Li, Junzhe Sun, Chu Wang, Serena Yeung-Levy, Felix Juefei-Xu

llms agents multimodal

arXiv AI

Jun 16

Propagating Structural Guidance: Synthesizing Fluorescein Angiography from Fundus Images and Sparse OCT Scans

arXiv:2606. 16234v1 Announce Type: cross Abstract: Fundus fluorescein angiography (FFA) is critical for assessing retinal vascular abnormalities, but its acquisition is invasive and not always feasible.

By Tengfei Ma, Ruiqi Wu, Chenran Zhang, Ye Geng, Na Su, Xiangyuan Duanmu, Tao Zhou, Yi Zhou, Wen Fan

multimodal benchmarks safety

arXiv AI

Jun 16

DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracing

arXiv:2606. 15796v1 Announce Type: cross Abstract: Mechanistic interpretability seeks to explain neural network behavior by decomposing model computations into interpretable features and circuits.

By Artyom Mazur, Nina Konovalova, Aibek Alanov

llms diffusion efficiency multimodal benchmarks safety

arXiv AI

Jun 16

XAI-Grounded Explanation Generation for Speech Deepfake Detection with Training-Free Multimodal Large Language Models

arXiv:2606. 16137v1 Announce Type: cross Abstract: Speech deepfake detection (SDD) systems require trustworthy explanations for reliable decision-making.

By Yupei Li, Qiyang Sun, Xiaoliang Wu, Chenxi Wang, Berrak Sisman, Bj\"orn W. Schuller

llms multimodal

arXiv Machine Learning

Jun 16

DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning

arXiv:2606. 15160v1 Announce Type: cross Abstract: Reasoning capabilities of multimodal large language models (MLLMs) have improved considerably in recent years.

By David Huang, Lianlei Shan

llms diffusion reinforcement-learning efficiency multimodal benchmarks

arXiv Machine Learning

Jun 16

Semi-Supervised Speech Confidence Detection using Pseudo-Labelling and Whisper Embeddings

arXiv:2606. 16505v1 Announce Type: cross Abstract: Understanding speaker confidence is crucial in educational settings, as it can enhance personalised feedback and improve learning outcomes.

By Adam Wynn, Jingyun Wang, Xiangyu Tan

rag multimodal

arXiv AI

Jun 16

Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models

arXiv:2606. 16808v1 Announce Type: new Abstract: While Large Reasoning Models (LRMs) excel at complex tasks, they remain highly vulnerable to sophisticated jailbreaks and direct harmful queries.

By Ke Miao, Jiaxin Li, Hongliang Chen, Yuke Hu, Zhan Qin

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jun 16

Scaling Human and G2P Supervision for Robust Phonetic Transcription

arXiv:2606. 16019v1 Announce Type: cross Abstract: Expert phonetic annotation is costly, especially for non-standard dialects and atypical speech.

By Alexander Metzger, Aruna Srivastava, Ruslan Mukhamedvaleev

multimodal benchmarks

arXiv AI

Jun 16

Fusion is not one-size-fits-all: Cross-Modal Representation Alignment for Time-to-Event Modeling

arXiv:2606. 15038v1 Announce Type: new Abstract: Accurate time-to-event (TTE) prediction from multimodal clinical data remains challenging due to modality imbalance and distribution shift.

By Zhemin Zhang, Weijie Chen, David Le, Amara Tariq, Alex Wallace, Matthew Stib, Juan Maria Farina, Chadi Ayoub, Reza Arsanjani, Imon Banerjee

diffusion multimodal safety

arXiv AI

Jun 16

Virtual Sensing to Enable Real-Time Monitoring of Inaccessible Locations & Unmeasurable Parameters

arXiv:2412. 00107v2 Announce Type: replace-cross Abstract: Real-time monitoring of safety-critical interior states remains an open problem in energy systems where physical instrumentation is infeasible.

By Kazuma Kobayashi, Farid Ahmed, Jaewan Park, Subhankar Sarkar, Souvik Chakraborty, Syed Bahauddin Alam

multimodal safety

arXiv AI

Jun 16

Attention, not scale, drives human-AI alignment in multimodal language prediction

arXiv:2308. 06035v4 Announce Type: replace Abstract: Humans routinely draw on visual context to predict upcoming words.

By Viktor Kewenig, Andrew Lampinen, Samuel A. Nastase, Christopher Edwards, Quitterie Lacome D'Elascombe, Akilles Rechardt, Jeremy I Skipper, Gabriella Vigliocco

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 16

DiRecT: Safe Diffusion-Based Planning via Receding-Horizon Denoising

arXiv:2606. 15359v1 Announce Type: new Abstract: Diffusion models have emerged as powerful tools for planning and control by learning multimodal distributions over actions and trajectories.

By Paolo Giaretta, Zeyang Li, Navid Azizan

diffusion multimodal benchmarks safety

arXiv AI

Jun 16

Multi-Modal Attention for Automated Disaster Damage Assessment Using Remote Sensing Imagery and Deep Learning

arXiv:2606. 14963v1 Announce Type: cross Abstract: Timely and accurate disaster damage assessment is crucial for effective emergency response, resource allocation, and recovery.

By Tewodros Syum Gebre, Jagrati Talreja, Leila Hashemi-Beni

multimodal

arXiv Machine Learning

Jun 16

SSNAPS: Audio-Visual Separation of Speech and Background Noise with Diffusion Inverse Sampling

arXiv:2602. 01394v2 Announce Type: replace-cross Abstract: This paper addresses the challenge of audio-visual single-microphone speech separation and enhancement in the presence of real-world environmental noise.

By Yochai Yemini, Yoav Ellinson, Rami Ben-Ari, Sharon Gannot, Ethan Fetaya

diffusion multimodal

arXiv AI

Jun 16

CAP: Towards PPG Universal Representation Learning with Patient-level Supervision

arXiv:2606. 15284v1 Announce Type: cross Abstract: Photoplethysmography (PPG) plays a central role in wearable health monitoring and clinical decision support.

By Chenyang He, Xinyi Shao, Shun Huang, Bosong Huang, Daoqiang Zhang, Ming Jing, Cheng Ding

multimodal benchmarks safety

arXiv AI

Jun 16

QoS-Aware Token Scheduling and Private Data Valuation for Multi-Modal Agentic Networks

arXiv:2606. 15573v1 Announce Type: new Abstract: In agentic systems, human-generated data records anchor the value of AI services.

By Yao Du, Jing Liu, Pengfei Xu, Zehua Wang, Victor C. M. Leung, Cyril Leung, Victoria Lemieux

agents multimodal benchmarks safety

arXiv AI

Jun 16

Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretability

arXiv:2606. 14758v1 Announce Type: cross Abstract: As Vision-Language Models are increasingly deployed in safety-critical applications, the trustworthiness of their explanations becomes crucial.

By Emirhan Bilgi\c{c}, Baptiste Caramiaux, Zhi Yan, Gianni Franchi

llms rag multimodal safety

Get the daily briefing