Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

4,022 stories · RSS feed

arXiv AI

Jul 14

ReflectWorld-MM: An Entity-Oriented Multi-Media Memory System for Open-Ended Video Streams

arXiv:2607. 09759v1 Announce Type: cross Abstract: Building assistants that can continually watch the world, remember what they see, and reason over their accumulated experience is a long-standing goal, and recently multimodal agents equipped with long-term memory over video streams have attracted increasing interest.

By Xiaokang Ma, Yifan Sun, Zhihong Jin, Jie Gu, Yudong Luo, Shenyi Shao, Chu Tang, Jingmin Chen, Li Pu

agents multimodal benchmarks data-engineering

arXiv Machine Learning

Jul 14

On the Efficiency of LoRA Fine-Tuning for Vision-Language-Action Models in Industrial Robotic Manipulation

arXiv:2607. 10172v1 Announce Type: cross Abstract: Deploying billion-parameter Vision-Language-Action (VLA) models on industrial hardware requires fine-tuning to bridge the embodiment gap.

By Finn Ferchau, Daniel Pommer, Cristian Axenie

robotics fine-tuning multimodal

arXiv AI

Jul 14

MAG: A Web-Agent Benchmark and Harness for Multimodal Action and Guide Generation

arXiv:2607. 10079v1 Announce Type: new Abstract: Digital Adoption Platforms (DAPs) are embedded overlays widely used on web systems to guide users through operations inside a page, helping them get started with unfamiliar interfaces quickly.

By Chengguang Gan, Hanjun Wei, Yunhao Liang, Zhixi Cai, Qinghao Zhang, Shiwen Ni

llms agents multimodal benchmarks

arXiv AI

Jul 14

DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs

arXiv:2607. 11228v1 Announce Type: cross Abstract: While Large Vision-Language Models (LVLMs) demonstrate remarkable capabilities, they remain highly susceptible to embedded social biases.

By Anqi Li, Jie Zhang, Zhongqi Wang, Songkai Xue, Jiahao Wang, Shiguang Shan, Xilin Chen

llms agents reinforcement-learning multimodal benchmarks safety

arXiv AI

Jul 14

Transcript-Free Lightweight Detection of Alzheimer's Disease from Spontaneous Speech Using Handcrafted MFCC-Dominant Acoustic Biomarkers

arXiv:2607. 10168v1 Announce Type: cross Abstract: It is still hard to find Alzheimer's disease (AD) early, especially when neuroimaging is expensive or tools that depend on language are not available.

By Rashin Gholijani Farahani, Azam Bastanfard

multimodal

arXiv AI

Jul 14

VoxENES 2026: Benchmarking Generalization of Speech Spoofing Detectors Against LLM-Era TTS and Voice Conversion

arXiv:2607. 11706v1 Announce Type: cross Abstract: Modern LLM-driven text-to-speech (TTS) and voice conversion (VC) systems produce synthetic speech that differs from the generators represented in many legacy spoofing benchmarks.

By Aastha Sharma, Guangjing Wang

llms fine-tuning multimodal benchmarks

arXiv AI

Jul 14

NVAITC AI Scientist: A Governed End-to-End Research System -- A Hypertension GWAS Case Study

arXiv:2607. 11084v1 Announce Type: new Abstract: Agentic research systems are emerging as a new paradigm for coordinating scientific workflows beyond isolated model inference, code generation, or statistical analysis.

By Eddie Huang (NVIDIA AI Technology Center, NVIDIA Corporation), Ken Liao (NVIDIA AI Technology Center, NVIDIA Corporation), Iven Fu (NVIDIA AI Technology Center, NVIDIA Corporation), Yang-Hsien Lin (NVIDIA AI Technology Center, NVIDIA Corporation), Chao-Shun Zhan (NVIDIA AI Technology Center, NVIDIA Corporation), Andy Liao (NVIDIA AI Technology Center, NVIDIA Corporation), Virginia Chen (NVIDIA AI Technology Center, NVIDIA Corporation), Johnson Sun (NVIDIA AI Technology Center, NVIDIA Corporation), Pika Wang (NVIDIA AI Technology Center, NVIDIA Corporation), Richard Huang (NVIDIA AI Technology Center, NVIDIA Corporation), Jiun-Cheng Jiang (NVIDIA AI Technology Center, NVIDIA Corporation), Ting-Yuan Liu (Department of Medical Research, China Medical University Hospital, Taichung, Taiwan, Master Program for Digital Health Innovation, China Medical University, Taichung, Taiwan), Hsing-Fang Lu (Department of Medical Research, China Medical University Hospital, Taichung, Taiwan, Laboratory for Statistical and Translational Genetics, RIKEN Center for Integrative Medical Sciences, Yokohama, Japan), Ray Y. Lee (AI-Driven Genomic Medicine and Drug Discovery Lab, China Medical University Hospital, Taichung, Taiwan), Chi-Chou Liao (Department of Medical Research, China Medical University Hospital, Taichung, Taiwan), Simon See (NVIDIA AI Technology Center, NVIDIA Corporation), Fuu-Jen Tsai (Department of Medical Research, China Medical University Hospital, Taichung, Taiwan, Department of Medical Laboratory Science and Biotechnology, Asia University, Taichung, Taiwan)

agents multimodal safety

arXiv Machine Learning

Jul 14

SpurLens: Automatic Detection of Spurious Cues in Multimodal LLMs

arXiv:2503. 08884v3 Announce Type: replace-cross Abstract: Unimodal vision models are known to rely on spurious correlations, but it remains unclear to what extent Multimodal Large Language Models (MLLMs) exhibit similar biases despite language supervision.

By Parsa Hosseini, Sumit Nawathe, Mazda Moayeri, Sriram Balasubramanian, Soheil Feizi

llms multimodal benchmarks safety

arXiv Machine Learning

Jul 14

NMIRacle: Multi-modal Generative Molecular Elucidation from IR and NMR Spectra

arXiv:2512. 19733v3 Announce Type: replace-cross Abstract: Molecular structure elucidation from spectroscopic data is a long-standing challenge in Chemistry, traditionally requiring expert interpretation.

By Federico Ottomano, Yingzhen Li, Alex M. Ganose

rag fine-tuning multimodal

arXiv AI

Jul 14

UNIT: Unleash Large Language Models Potential for Graph Continual Learning

arXiv:2607. 10159v1 Announce Type: new Abstract: In real-world multimodal web scenarios, graph-structured data often arrives in a streaming manner, making graph continual learning a crucial paradigm for continuously modeling such evolving structures.

By Tairan Huang, Yili Wang, Beibei Hu, Yiting Shi, Qiutong Li, Changlong He, Jianliang Gao

llms fine-tuning multimodal benchmarks

arXiv AI

Jul 14

AutoVSR: Automatic Visual-to-Symbolic Reasoning for Symbolic Expression Generation from Circuit Schematic

arXiv:2607. 11338v1 Announce Type: new Abstract: Symbolic expressions can effectively characterize and predict circuit behavior, but deriving them directly from circuit schematics is challenging.

By Zhe Xiao, Longfei Li, Xu He, Haoying Wu, Zixing Zhang, Mingyu Liu

llms agents multimodal benchmarks

arXiv Machine Learning

Jul 14

LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models

arXiv:2607. 11257v1 Announce Type: cross Abstract: Pathology Foundation Models (PFMs) offer powerful Whole Slide Image (WSI) representations but suffer from massive computational costs.

By Gangsu Kim, Won-Ki Jeong

nlp efficiency multimodal safety

arXiv Machine Learning

Jul 14

Multimodal rumor detection enhanced by external evidence and forgery features

arXiv:2601. 14954v3 Announce Type: replace Abstract: Social media increasingly disseminates information through mixed image text posts, but rumors often exploit subtle inconsistencies and forged content, making detection based solely on post content difficult.

By Han Li, Hua Sun

llms multimodal safety

arXiv AI

Jul 14

AdvNav: Behavior-Guided Black-Box Adversarial Attacks on Vision-Language Navigation

arXiv:2607. 11063v1 Announce Type: new Abstract: Despite progress in Embodied AI, Vision-and-Language Navigation systems remain vulnerable to adversarial visual disturbances.

By Chenyang Li, Kaige Li, Zeyu Jiang, Changhao Chen

llms agents robotics multimodal safety

arXiv Machine Learning

Jul 14

CHM-Net: Center Heatmap-driven Macro-Micro Modeling Network for MRI-based Microbial Density Stratification

arXiv:2607. 09812v1 Announce Type: cross Abstract: Microbial density is clinically important for tumor assessment and treatment decision-making, and recent advances in deep learning suggest that it can be non-invasively inferred from multimodal MRI.

By Jiaming Liang, Haolin Chen, Tingting Li, Bowen Yu, Qianyan Long, Tinghe Zhang, Xi Zhong, Xiaowei Hu, Xiaoqi Sheng, Hongmin Cai

computer-vision multimodal

arXiv AI

Jul 14

Omni-Decision: A Progressive Evidence-State Agent System for Omni-Modal QA

arXiv:2607. 11433v1 Announce Type: new Abstract: Omni-modal evidence-seeking QA requires agents to answer questions whose evidence is sparsely distributed across videos, audio, images, web pages, and computation results.

By Ming Ma, Yi Zhu, Yiran Zhong, Feida Zhu, Weigao Sun, Junhan Shi, Lingrui Mei, Tianming Yang, Steven Hoi

agents multimodal

arXiv AI

Jul 14

BackgroundMellow: A Multi-Modal Cohesive Framework for Narrative-Driven Rich Cinematic Soundscape Generation

arXiv:2607. 11364v1 Announce Type: cross Abstract: Generating immersive, synchronized and cinematic audio for long-form textual narratives remains a significant challenge in multi-modal AI.

By Ajitesh Jamulkar, Aritra Hazra

agents diffusion nlp multimodal safety

arXiv AI

Jul 14

The evolution of AI from image interpretation toward scientific inference in nanoparticle electron microscopy

arXiv:2607. 10388v1 Announce Type: cross Abstract: Artificial intelligence (AI) is transforming electron microscopy by enabling quantitative analysis of increasingly large and complex datasets for nanoparticle characterization.

By Evropi Toulkeridou, Jiafei Li, Leonardo Lari, Panagiotis Grammatikopoulos

llms agents computer-vision multimodal benchmarks

arXiv Machine Learning

Jul 14

CoGenCast: A Coupled Autoregressive-Flow Generative Framework for Time Series Forecasting

arXiv:2602. 03564v2 Announce Type: replace Abstract: Time series forecasting can be viewed as a generative problem that requires both semantic understanding over contextual conditions and stochastic modeling of continuous temporal dynamics.

By Mingyue Cheng, Yaguo Liu, Daoyu Wang, Xiaoyu Tao, Qi Liu

llms diffusion multimodal benchmarks

arXiv Machine Learning

Jul 14

MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search

arXiv:2607. 11030v1 Announce Type: cross Abstract: Multimodal information is pivotal for e-commerce search ranking.

By Zhen-Lin Chen, Maosen Sheng, Peng Lin, Jianmin Chen, Zhuojian Xiao, Dongyue Wang, Xiwei Zhao

llms fine-tuning multimodal

Get the daily briefing