Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

7,813 stories · RSS feed

arXiv AI

Sep 30

Suppression Is Not Forgetting: Residual Recoverability in Visual Concept Unlearning for VLMs

arXiv:2604.03114v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) may need to forget visual concepts after deployment because of privacy, copyright, licensing, safety, or policy...

By Zhangyun Tan, Zeliang Zhang, Jiani Liu, Susan Liang, Yolo Y. Tang, Lisha Chen, Chenliang Xu

llms multimodal safety

arXiv AI

Sep 30

Trojan Hippo Bench: A Dynamic Benchmark for Persistent Memory Attacks and Defenses in LLM Agents

arXiv:2605.01970v4 Announce Type: replace-cross Abstract: Memory systems enable otherwise stateless LLM agents to persist user information across sessions, but also introduce a new attack surface. Th...

By Debeshee Das, Julien Piet, Darya Kaviani, Luca Beurer-Kellner, Florian Tram\`er, David Wagner

llms rag agents multimodal benchmarks safety

arXiv AI

Sep 30

TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception

arXiv:2609.32013v2 Announce Type: replace-cross Abstract: We present TriO, a multi-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow and LiDAR. In contrast to pri...

By Quinlan Sykora, Sourav Biswas, Christopher Diehl, Andrew Cunningham, Thomas Gilles, Raquel Urtasun

computer-vision multimodal benchmarks

arXiv AI

Sep 30

Multimodal LLMs Outperform Pathology Foundation Models in Cross-Domain Histological Similarity

arXiv:2609.32876v2 Announce Type: replace-cross Abstract: State-of-the-art pathology foundation models, trained on millions of histology tiles, can fail to preserve tissue similarity when comparisons...

By Yishu Zhang, Yun Li, Daiwei Zhang

llms multimodal benchmarks

arXiv AI

Sep 30

F4R: Failure-Driven Recognition, Reconstruction, Refinement, and Redeployment for Continual Robot Self-Improvement

arXiv:2609.35575v2 Announce Type: replace-cross Abstract: The real-world performance of current vision-language-action models is fundamentally constrained by the limited coverage of expert demonstrat...

By Zhuoyuan Yu, Jiacheng Wang, Tianle Liu, Yihua Ren, Peng Yu, Chen Bai, Ziheng Zhang, Yufei Jia, Jindou Jia, Yuhang Zhang, Xinrui Zhang, Shang Yujing, Yuxiang Chen, Chuhao Zhou, Tiancai Wang, Jianfei Yang

agents reinforcement-learning robotics multimodal

arXiv Machine Learning

Sep 30

Koa-action: Fast and Consistent Structured Decision Making with Generative LLMs

arXiv:2609.36115v1 Announce Type: new Abstract: Industry applications often demand low-latency classification, yet current large language model (LLM) approaches remain poorly suited for latency-criti...

By Shenghong Dai, Shiva Kumar Pentyala, Yingchi Liu, Shubham Mehrotra, Suman Banerjee, James Zhu, Bin Bi, Sitaram Asur, Phil Mui

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

Multi-Agent Flow Matching with Decoupled Generative Guidance

arXiv:2609.38133v1 Announce Type: new Abstract: Generative modeling is widely used for producing diverse objects from complex, multimodal distributions. However, its expressivity does not, in general...

By Ruoyu Lin, Magnus Egerstedt, Fabio Pasqualetti

agents diffusion robotics multimodal

arXiv Machine Learning

Sep 30

Distill Locally, Schedule Globally: Flow Maps for Few-Step Text-to-Speech

arXiv:2609.36324v1 Announce Type: cross Abstract: Flow-matching text-to-speech (TTS) models achieve high synthesis quality but require many neural function evaluations (NFEs) to integrate their gener...

By Yentl Collin, Evan Dufraisse, Amr Mohamed, Amine Khelif Khelif, Dani Bouch, Guokan Shang

efficiency multimodal safety

arXiv Machine Learning

Sep 30

Disentangling Spurious Correlations in Vision-Language-Action Models via Predicting Domain-Invariant Latent Lookahead

arXiv:2609.37165v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models remain brittle under visual distribution shifts, often relying on spurious correlations tied to domain-specific f...

By Junghyun Kim, Ngseo Kim, ChungWoo Lee, Seoyeon Lee, Woo-Jeong Baek, Adam Zhou, Chip Huyen, Jun-Ki Lee, Gi-Cheon Kang, Byoung-Tak Zhang

robotics multimodal

arXiv Machine Learning

Sep 30

Seeing Is Not Addressing: Auditing Linguistic Access to Frozen Visual Geometry

arXiv:2609.37230v1 Announce Type: cross Abstract: Visual distinctions are often finer than those reflected in linguistic conceptualization. Vision-language models exhibit a similar asymmetry: a disti...

By Woosang Jeon, Jiwon Yang, Soo Chung, Taehyeong Kim

llms diffusion multimodal safety

arXiv Machine Learning

Sep 30

Backdoor Mitigation in Decentralized LLM Fine-Tuning

arXiv:2609.37367v1 Announce Type: cross Abstract: Decentralized large language model (LLM) fine-tuning lets organizations collaboratively train a shared LLM on data they cannot pool, without a centra...

By Sayan Biswas, Jade Garcia Bourr\'ee, Rachid Guerraoui, Maxime Jacovella, Anne-Marie Kermarrec, Sathwika Peechara, Martijn de Vos, Milos Vujasinovic

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

PoE-Fuse: Precision-Weighted Expert Fusion for Bi-Temporal Change Understanding

arXiv:2609.37485v1 Announce Type: cross Abstract: Bi-temporal change understanding, which localizes and characterizes what changed between two satellite images, is central to disaster response and en...

By Haruki Watase, Shunya Nagashima, Takayuki Nishimura

llms reinforcement-learning fine-tuning multimodal

arXiv Machine Learning

Sep 30

FORUM: Frozen Outputs Reconciled Using Model Agreement for Visual Grounding

arXiv:2609.37488v1 Announce Type: cross Abstract: Frozen multimodal large language models (MLLMs) now solve standard referring expression comprehension with a single prompted call, yet on adversarial...

By Taiyo Sato, Takamasa Sanda, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, Shunya Nagashima

llms multimodal benchmarks safety

arXiv Machine Learning

Sep 30

SYNCR: Diagnosing and Learning Cross-Video Reasoning from Simulation

arXiv:2609.37918v1 Announce Type: cross Abstract: Reasoning across videos requires aligning events, matching identities, comparing motion, and integrating partial observations. Evaluating these capab...

By Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Sep 30

Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems

arXiv:2602.15382v3 Announce Type: replace-cross Abstract: Heterogeneous multi-agent systems combine models with different capabilities through a common communication interface. Exchanging internal st...

By Xiaoze Liu, Ruowang Zhang, Weichen Yu, Siheng Xiong, Liu He, Feijie Wu, Hoin Jung, Matt Fredrikson, Xiaoqian Wang, Jing Gao

llms agents efficiency multimodal benchmarks safety

arXiv Machine Learning

Sep 30

OpenTumorBoard: A Real-World Benchmark of Multidisciplinary Tumor Board Discussion Trajectories

arXiv:2609.32810v2 Announce Type: replace-cross Abstract: Multidisciplinary tumor boards integrate multimodal clinical observations and longitudinal patient histories through specialist discussions,...

By Anqi Li, Zhixuan Ge, Yixuan Duan, Jiarong Qian, Chi-Yu Chen, MingYu Lu, Huan-Yu Hsu, Yu Gu, Yue Guo, Sheng Wang, Wei Qiu, Hanwen Xu

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv Computation and Language

Sep 30

FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech

arXiv:2609.35791v1 Announce Type: new Abstract: Natural turn-taking in full-duplex voice interaction requires determining from partial speech whether a pause reflects hesitation or a completed conver...

By Puneet Mathur, Dinesh Manocha

llms multimodal

arXiv Computation and Language

Sep 30

Can We Still Trust Disaster Social Sensing? Empirical Evidence on Detecting AI-Generated Social Media Posts

arXiv:2609.35821v1 Announce Type: new Abstract: Disaster social sensing converts public social-media posts into evidence for situational awareness and humanitarian needs, but generative artificial in...

By Xiaoshan Zhou, Zaifu Zhan

llms multimodal

arXiv Computation and Language

Sep 30

Video2Skill: From Streaming Experience to Reusable Embodied Skills

arXiv:2609.36691v1 Announce Type: new Abstract: Manipulation behaviors vary widely across objects and scenes, but they share a small set of reusable skills, and planning with these skills helps embod...

By Jianshu Zhang, Ce Zhang, Xiyuan Yang, Chenwei Xu, Haoran Lu, Yijiang Li, Yaqi Xie, Katia P. Sycara, Han Liu

llms agents robotics fine-tuning multimodal benchmarks

arXiv Computation and Language

Sep 30

Rethinking Multimodal Fake News Detection in the Generative AI Era

arXiv:2609.36850v1 Announce Type: new Abstract: Generative content is increasingly entering the production and dissemination of news, transforming fake news from manually fabricated or simply manipul...

By Wenbin Shen, Guoxuan Qin, Guangxu Yao, Baodong Wang, Yuanbo Rui, Zhichao Lian

multimodal benchmarks

Get the daily briefing