Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

7,971 stories · RSS feed

arXiv AI

Sep 30

Multimodal LLMs Outperform Pathology Foundation Models in Cross-Domain Histological Similarity

arXiv:2609.32876v2 Announce Type: replace-cross Abstract: State-of-the-art pathology foundation models, trained on millions of histology tiles, can fail to preserve tissue similarity when comparisons...

By Yishu Zhang, Yun Li, Daiwei Zhang

llms multimodal benchmarks

arXiv AI

Sep 30

F4R: Failure-Driven Recognition, Reconstruction, Refinement, and Redeployment for Continual Robot Self-Improvement

arXiv:2609.35575v2 Announce Type: replace-cross Abstract: The real-world performance of current vision-language-action models is fundamentally constrained by the limited coverage of expert demonstrat...

By Zhuoyuan Yu, Jiacheng Wang, Tianle Liu, Yihua Ren, Peng Yu, Chen Bai, Ziheng Zhang, Yufei Jia, Jindou Jia, Yuhang Zhang, Xinrui Zhang, Shang Yujing, Yuxiang Chen, Chuhao Zhou, Tiancai Wang, Jianfei Yang

agents reinforcement-learning robotics multimodal

arXiv Machine Learning

Sep 30

Koa-action: Fast and Consistent Structured Decision Making with Generative LLMs

arXiv:2609.36115v1 Announce Type: new Abstract: Industry applications often demand low-latency classification, yet current large language model (LLM) approaches remain poorly suited for latency-criti...

By Shenghong Dai, Shiva Kumar Pentyala, Yingchi Liu, Shubham Mehrotra, Suman Banerjee, James Zhu, Bin Bi, Sitaram Asur, Phil Mui

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

Multi-Agent Flow Matching with Decoupled Generative Guidance

arXiv:2609.38133v1 Announce Type: new Abstract: Generative modeling is widely used for producing diverse objects from complex, multimodal distributions. However, its expressivity does not, in general...

By Ruoyu Lin, Magnus Egerstedt, Fabio Pasqualetti

agents diffusion robotics multimodal

arXiv Machine Learning

Sep 30

Distill Locally, Schedule Globally: Flow Maps for Few-Step Text-to-Speech

arXiv:2609.36324v1 Announce Type: cross Abstract: Flow-matching text-to-speech (TTS) models achieve high synthesis quality but require many neural function evaluations (NFEs) to integrate their gener...

By Yentl Collin, Evan Dufraisse, Amr Mohamed, Amine Khelif Khelif, Dani Bouch, Guokan Shang

efficiency multimodal safety

arXiv Machine Learning

Sep 30

Disentangling Spurious Correlations in Vision-Language-Action Models via Predicting Domain-Invariant Latent Lookahead

arXiv:2609.37165v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models remain brittle under visual distribution shifts, often relying on spurious correlations tied to domain-specific f...

By Junghyun Kim, Ngseo Kim, ChungWoo Lee, Seoyeon Lee, Woo-Jeong Baek, Adam Zhou, Chip Huyen, Jun-Ki Lee, Gi-Cheon Kang, Byoung-Tak Zhang

robotics multimodal

arXiv Machine Learning

Sep 30

Seeing Is Not Addressing: Auditing Linguistic Access to Frozen Visual Geometry

arXiv:2609.37230v1 Announce Type: cross Abstract: Visual distinctions are often finer than those reflected in linguistic conceptualization. Vision-language models exhibit a similar asymmetry: a disti...

By Woosang Jeon, Jiwon Yang, Soo Chung, Taehyeong Kim

llms diffusion multimodal safety

arXiv Machine Learning

Sep 30

Backdoor Mitigation in Decentralized LLM Fine-Tuning

arXiv:2609.37367v1 Announce Type: cross Abstract: Decentralized large language model (LLM) fine-tuning lets organizations collaboratively train a shared LLM on data they cannot pool, without a centra...

By Sayan Biswas, Jade Garcia Bourr\'ee, Rachid Guerraoui, Maxime Jacovella, Anne-Marie Kermarrec, Sathwika Peechara, Martijn de Vos, Milos Vujasinovic

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

PoE-Fuse: Precision-Weighted Expert Fusion for Bi-Temporal Change Understanding

arXiv:2609.37485v1 Announce Type: cross Abstract: Bi-temporal change understanding, which localizes and characterizes what changed between two satellite images, is central to disaster response and en...

By Haruki Watase, Shunya Nagashima, Takayuki Nishimura

llms reinforcement-learning fine-tuning multimodal

arXiv Machine Learning

Sep 30

FORUM: Frozen Outputs Reconciled Using Model Agreement for Visual Grounding

arXiv:2609.37488v1 Announce Type: cross Abstract: Frozen multimodal large language models (MLLMs) now solve standard referring expression comprehension with a single prompted call, yet on adversarial...

By Taiyo Sato, Takamasa Sanda, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, Shunya Nagashima

llms multimodal benchmarks safety

arXiv Machine Learning

Sep 30

SYNCR: Diagnosing and Learning Cross-Video Reasoning from Simulation

arXiv:2609.37918v1 Announce Type: cross Abstract: Reasoning across videos requires aligning events, matching identities, comparing motion, and integrating partial observations. Evaluating these capab...

By Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems

arXiv:2602.15382v3 Announce Type: replace-cross Abstract: Heterogeneous multi-agent systems combine models with different capabilities through a common communication interface. Exchanging internal st...

By Xiaoze Liu, Ruowang Zhang, Weichen Yu, Siheng Xiong, Liu He, Feijie Wu, Hoin Jung, Matt Fredrikson, Xiaoqian Wang, Jing Gao

llms agents efficiency multimodal benchmarks safety

arXiv Machine Learning

Sep 30

OpenTumorBoard: A Real-World Benchmark of Multidisciplinary Tumor Board Discussion Trajectories

arXiv:2609.32810v2 Announce Type: replace-cross Abstract: Multidisciplinary tumor boards integrate multimodal clinical observations and longitudinal patient histories through specialist discussions,...

By Anqi Li, Zhixuan Ge, Yixuan Duan, Jiarong Qian, Chi-Yu Chen, MingYu Lu, Huan-Yu Hsu, Yu Gu, Yue Guo, Sheng Wang, Wei Qiu, Hanwen Xu

llms reinforcement-learning fine-tuning multimodal benchmarks safety