Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

8,098 stories · RSS feed

arXiv AI

Sep 15

Spatial Reasoning via Modality Switching Between Language and Symbolic Representations

arXiv:2606.31285v3 Announce Type: replace Abstract: Human reasoning is inherently multimodal: when problems become difficult, we rarely think in words alone. We often externalize our reasoning by ske...

By Shreya Rajpal, Tanawan Premsri, Parisa Kordjamshidi

llms multimodal

arXiv AI

Sep 15

Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answering

arXiv:2604.27724v2 Announce Type: replace Abstract: Medical retrieval-augmented generation (RAG) systems typically operate on text chunks extracted from biomedical literature, discarding the rich vis...

By Xupeng Chen, Binbin Shi, Chenqian Le, Jiaqi Zhang, Kewen Wang, Ran Gong, Jinhan Zhang, Chihang Wang

llms rag computer-vision nlp multimodal benchmarks

arXiv AI

Sep 15

MARCUS: An agentic, multimodal vision-language model for cardiac diagnosis and management

arXiv:2603.22179v2 Announce Type: replace Abstract: Cardiovascular disease remains the leading cause of global mortality, with progress hindered by human interpretation of complex cardiac tests. Curr...

By Jack W O'Sullivan, Mohammad Asadi, Lennart Elbe, Akshay Chaudhari, Tahoura Nedaee, Francois Haddad, Ivan Lopez, Fang Cao, Michael Salerno, Li Fe-Fei, Ehsan Adeli, Rima Arnaout, Euan A Ashley

llms agents multimodal benchmarks

arXiv AI

Sep 15

Modality-Guided Mixture of Structured Experts with Entropy-Triggered Routing for Multimodal Recommendation

arXiv:2602.20723v3 Announce Type: replace Abstract: Multimodal recommenders combine collaborative behavior with visual and textual item evidence, whose usefulness varies across user-item interactions...

By Ji Dai, Quan Fang, DeSheng Cai

multimodal benchmarks safety

arXiv AI

Sep 15

Circuit-MLLM: Topological Logic-Guided Latent-Space Visual Reasoning for Circuit Schematic Understanding

arXiv:2609.15668v1 Announce Type: cross Abstract: Through pre-training on extensive text and image datasets, current multi-modal large language models (MLLMs) achieve strong performance on general ta...

By Jinyuan Deng, Yuqi Jiang, Wenjing Huang, Xin Li, Qi Sun, Cheng Zhuo

llms diffusion multimodal

arXiv AI

Sep 15

A Unified Vision-Language Model for PSMA PET/CT Report Generation, Visual Question Answering, and Lesion Segmentation

arXiv:2609.15603v1 Announce Type: cross Abstract: Accurate PSMA PET/CT interpretation is central to prostate cancer management, yet existing PET/CT AI models typically address isolated tasks. We prop...

By Yang Xing, Jiong Wu, Savas Ozdemir, Yang Zhou, Boxiao Yu, Ying Zhang, Zheren Zhu, Chenyu You, Wei Shao, Yang Lu, Kang Wang, Tinsu Pan, Yang Yang, Kuang Gong

llms computer-vision nlp fine-tuning multimodal safety

arXiv AI

Sep 15

Open-UniMo: Towards Unified Motion-Language Understanding and Generation in the Open World

arXiv:2609.14615v1 Announce Type: cross Abstract: Unified motion generation and understanding is crucial for embodied AI systems that can both synthesize and interpret human actions in open-world env...

By Guocun Wang, Kenkun Liu, Guorui Song, Jing Lin, Zhe Huang, Luyuan Zhang, Dake Zhong, Choo Sin Wai, Xiaoguang Han, Haoqian Wang

llms robotics fine-tuning multimodal benchmarks safety

arXiv AI

Sep 15

ReWeight: Leveraging Human Data for VLA Post-Training via Demonstration Retrieval and Sample Weighting

arXiv:2609.13851v1 Announce Type: cross Abstract: Post-training vision-language-action (VLA) models for specific robots and tasks requires in-domain demonstrations, yet collecting diverse robot data...

By Chenwei Wang, Dianye Huang, Match W. L. Ko, Chenjia Bai, Zhongliang Jiang

robotics multimodal

arXiv AI

Sep 15

Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings

arXiv:2609.15296v1 Announce Type: new Abstract: Universal multimodal embedding (UME) learns unified representations across modalities, enabling a single model to support diverse retrieval tasks. Rece...

By Mingzhou Jiang, Peixi Wu, Hang Cheng, Yunhao Zhou, Biao Yang, Wei Yuan, Yun Li, Fan Yang, Wenwu Ou, Honghui He

rag efficiency multimodal benchmarks

arXiv AI

Sep 15

TimeThink: Eliciting Compositional Reasoning in Timeseries Large Language Models

arXiv:2609.13457v1 Announce Type: new Abstract: Timeseries multimodal large language models (TS-MLLMs) have recently begun leveraging the reasoning capabilities of large language models (LLMs) for qu...

By Sudarshan Regmi, Arvind Pillai, Yu Yvonne Wu, Yuliang Chen, Bibek Panthi, Tess Z. Griffin, Michael V. Heinz, Lisa Marsch, Nicholas C. Jacobson, Andrew Campbell

llms reinforcement-learning multimodal benchmarks

arXiv Computer Vision

Sep 15

ReMemNav: Memory-Based Decision Correction and Target Verification for Zero-Shot Object Navigation

arXiv:2603.26788v3 Announce Type: replace-cross Abstract: Zero-shot object navigation requires agents to locate unseen targets in unfamiliar environments without prior maps or task-specific training....

By Feng Wu, Wei Zuo, Wenliang Yang, Jun Xiao, Yang Liu, Xinhua Zeng

llms agents multimodal

arXiv Computer Vision

Sep 15

VIGIL: Tackling Hallucination Detection in Image Recontextualization

arXiv:2602.14633v3 Announce Type: replace Abstract: We introduce VIGIL (Visual Inconsistency & Generative In-context Lucidity), a benchmark dataset and framework that provides a fine-grained categori...

By Joanna Wojciechowicz, Maria {\L}ubniewska, Jakub Antczak, Justyna Baczy\'nska, Wojciech Gromski, Wojciech Koz{\l}owski, Maciej Zieba

multimodal benchmarks safety

arXiv Computer Vision

Sep 15

Hypergraph-Regularized Gramian Volumes for Multimodal Retrieval

arXiv:2609.15320v1 Announce Type: new Abstract: Volume-based multimodal retrieval jointly scores a text query with a candidate's video, audio, and subtitle embeddings. While this approach captures hi...

By Anindya Nag, Ambuj Mehrish, Sebastiano Vascon

rag multimodal benchmarks safety

arXiv AI

Sep 15

Parameter-Efficient Adaptation of Pretrained Language Models for Time-Series Forecasting

arXiv:2609.15344v1 Announce Type: new Abstract: We study the adaptation of pretrained language models to univariate time-series forecasting through a parameter-efficient transfer learning framework,...

By Tamanna Kumavat, Georg Brunner, Kyriakos Flouris

llms rag nlp fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 15

Perceive, Refine, Reason: A Calibrated Pipeline for Measuring Indicators in Strategic Visual Communication on Social Media

arXiv:2609.14699v1 Announce Type: new Abstract: Visual content shapes audience perception and opinion on social media, and computational social science increasingly relies on automated tools to analy...

By Weihong Qi, Chen Ling

llms multimodal

arXiv AI

Sep 15

From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivity

arXiv:2609.10714v1 Announce Type: cross Abstract: The ambitious requirements of sixth-generation (6G) networks are driving communication systems from reliable bit delivery toward meaning-aware and ta...

By Yu Ma, Zhen Gao, Li Qiao, Xiaoyuan Zhang, Mahdi Boloursaz Mashhadi, Yin Xu, Wenjun Xu, Xiaodong Xu, Kaibin Huang, Jiangzhou Wang, Rahim Tafazolli, Sheng Chen, Tony Q. S. Quek, Ping Zhang

agents robotics multimodal

arXiv AI

Sep 15

ShieldVLA: Feasibility-Aware Safety Alignment for Vision-Language-Action Models

arXiv:2609.13231v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models demonstrate strong generalization in robotic manipulation and navigation, but existing fine-tuning methods provid...

By Manan Tayal, Akshay Nambi

robotics fine-tuning multimodal benchmarks safety

arXiv AI

Sep 15

TryOnReward: Learning Foveated Consistency for Reinforcement Fine-Tuning of Virtual Try-On

arXiv:2609.13259v1 Announce Type: cross Abstract: Virtual Try-On (VTON) aims to dress a person with the reference garment, producing visually reasonable results aligned with human preferences. Turnin...

By Xueheng Li, Yong Liu, Xiaolong Fu, Wen Xue, Chengjun Xie, Yipeng Sun, Yan Li, Simiu Gu

reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Sep 15

LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI Agents

arXiv:2609.13287v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) achieve high decoding efficiency through block-parallel, arbitrary-order generation, making them attractive f...

By Zhangxuan Gu, Haoxing Chen, Qi Qin, Yi Xin, Kai Gan, Lin Liu, Long Cui, Xiaomei Wang, Beitong Zhou, Yunzhu Zhang, Zhengwen Zeng, Changlong Gao, Weizhi Chen, Rongchao Zhang, Haoyuan Wu, Shuheng Shen, Changhua Meng, Weiqiang Wang, Jianguo Li, Zhenzhong Lan

llms agents diffusion fine-tuning multimodal benchmarks

arXiv AI

Sep 15

When Malicious Instructions Persist: Persistent Memory Poisoning Attack on Harness-Based Agents

arXiv:2609.13889v1 Announce Type: cross Abstract: Harness design has transformed the development of LLM-based agents by integrating memory, tool use, and runtime control. However, this design also in...

By Shuhuai Huang, Jingfeng Zhang, Hong Jia

llms agents multimodal safety

Get the daily briefing