Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

7,813 stories · RSS feed

arXiv Computation and Language

Oct 1

Recovering Off-Policy Supervision for Speculative Decoding

arXiv:2609.38795v1 Announce Type: new Abstract: Block drafters for speculative decoding are commonly trained on corpora written by external models, where a single off-policy token invalidates supervi...

By Jungseob Lee, Chanjun Park, Sugyeong Eo, Hyeonseok Moon

multimodal

arXiv Computation and Language

Oct 1

Fairness Beyond a Single Run: Training-Seed Variability in Speech LLM Adaptation

arXiv:2609.38976v1 Announce Type: new Abstract: Demographic fairness gaps in automatic speech recognition are almost always reported from a single training run. We fine-tune the Q-former projector an...

By Srishti Ginjala, Eric Fosler-Lussier, Srinivasan Parthasarathy

llms fine-tuning multimodal safety

arXiv Computation and Language

Oct 1

4MT-VLM: How Coarse Is a VLMs Cognitive Map?

arXiv:2609.39238v1 Announce Type: new Abstract: An agent that moves must recognise a place from a viewpoint it has never seen. We introduce 4MT-VLM, a dataset of procedurally generated landscapes, ea...

By Markus Frey

llms agents multimodal benchmarks

arXiv Computation and Language

Oct 1

Spike-driven Vision-Language-Action Model

arXiv:2609.39514v1 Announce Type: new Abstract: Vision-language-action (VLA) models bridge multimodal understanding and robotic control, advancing the dominant paradigm for embodied intelligence. How...

By Shuai Wang, Malu Zhang, Mingquan Liu, Weihui Dai, Dehao Zhang, Jieyuan Zhang, Yimeng Shan, Zijian Zhou, Yang Yang

llms rag robotics multimodal

arXiv Computer Vision

Oct 1

PhyProbe: Rethinking Physical Consistency Evaluation in Generated Videos

arXiv:2609.38377v1 Announce Type: new Abstract: Evaluating the physical consistency of generated videos remains a fundamental challenge. Existing approaches rely on off-the-shelf vision-language mode...

By Max Ku, Jiaojiao Fan, Zekun Hao, Francesco Ferroni, Heng Wang, Wenhu Chen, Ming-Yu Liu, Prithvijit Chattopadhyay

llms fine-tuning multimodal benchmarks

arXiv Computer Vision

Oct 1

Team MSU GenText-Forensics Challenge 2026 Technical Report

arXiv:2609.38391v1 Announce Type: new Abstract: Document text forgery has evolved beyond simple pixel-level manipulation: modern attacks alter not only the appearance of a document but also its meani...

By Kirill Koltsov, Aleksandr Gushchin, Dmitriy Vatolin, Anastasia Antsiferova

llms computer-vision robotics fine-tuning efficiency multimodal

arXiv Computer Vision

Oct 1

MOBA-VL: Event-Localized Multi-Turn Reinforcement Learning for Real-Time MOBA Commentary

arXiv:2609.38428v1 Announce Type: new Abstract: Real-time commentary for Multiplayer Online Battle Arena (MOBA) esports requires a vision-language model (VLM) to narrate a live match second by second...

By Shengyun Zhong, Xinkang Zhao, Ziyuan Chu, Linchao Zhu

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv Computer Vision

Oct 1

Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Models

arXiv:2609.38485v1 Announce Type: new Abstract: Unified multimodal models (UMMs) train image understanding and autoregressive image generation on shared parameters, and the two objectives are known t...

By Shuyang Jiang, Fucheng Deng, Yuchuan Luo, Zhenyu Wu

diffusion multimodal

arXiv Computer Vision

Oct 1

Detail in Context: A Dual-Scale Machine Learning Framework for Mycosis Fungoides Detection

arXiv:2609.38560v1 Announce Type: new Abstract: Mycosis fungoides (MF) is a rare form of cutaneous T-cell lymphoma that is often misdiagnosed in early stages due to its visual similarity to benign in...

By Mohamed Hazem, Tarek Waleed, Omar Khaled, Nada Omar, Mahmoud Raslan, Marwa Mohamed Fawzy, Aya Fahim, Rania M. Mogawer, Ahmed Mourad, Kariman Mansour, Muhammad Rushdi

multimodal

arXiv Computer Vision

Oct 1

Vision-Language-Action Autonomous Driving Agent with Language-based Memory

arXiv:2609.38641v1 Announce Type: new Abstract: Vision-Language-Action (VLA) foundation models have recently emerged as one of the prevailing solutions for autonomous driving, as they can utilize kno...

By Kai Yan, Xiangyu Chen, Yulong Cao, Alex Naumann, Peter Karkus, Yan Wang, Jef Packer, Alex Schwing, Yuxiong Wang, Boris Ivanovic, Wenjie Luo, Marco Pavone

agents nlp reinforcement-learning robotics fine-tuning multimodal

arXiv Computer Vision

Oct 1

MEMO: Multi-Level Entity-Aware Memory for Streaming Video Understanding

arXiv:2609.38900v1 Announce Type: new Abstract: Streaming video understanding requires models to process unbounded visual streams while preserving rich visual semantics across vast temporal horizons,...

By Yinying Li, Yuqian Fu, Yulin Dai, Jingyu Gong, Tianwen Qian, Xiaoling Wang

llms multimodal benchmarks

arXiv Computer Vision

Oct 1

From Image Interpretation to Clinical Reasoning: Upstream Physician-Context-Aware Multimodal Learning with Causal Reinforcement Learning

arXiv:2609.38924v1 Announce Type: new Abstract: Major adverse cardiovascular events (MACE) remain the leading cause of mortality worldwide. Opportunistic screening using routinely acquired clinical d...

By Jialu Pi, Yanan Ma, Weijie Chen, Owen Crystal, Shubham Trivedi, Stephen Xie, Anna Silverman, Matthew Stib, Chadi Ayoub, Reza Arsanjani, Imon Banerjee

llms reinforcement-learning efficiency multimodal benchmarks

arXiv Computer Vision

Oct 1

When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion

arXiv:2609.39004v1 Announce Type: new Abstract: Multimodal image fusion (MMIF) aims to integrate complementary information from different modalities into a high-quality fused image and support downst...

By Zeyu Wang, Jiayu Wang, Haiyu Song, Haoran Duan

multimodal benchmarks

arXiv Computer Vision

Oct 1

TSMD: Temporal-Stream Modality Dropout for Robust Video Highlight Detection

arXiv:2609.39051v1 Announce Type: new Abstract: Existing multimodal video highlight detectors typically assume that visual, audio, and textual streams are continuously available. In practice, however...

By Bo-Yuan Cheng, Kuan-Yu Chen, Po-Han Huang, Jeng-Lin Li, Jian-Jiun Ding

multimodal

arXiv Computer Vision

Oct 1

Agentic Tool-Augmented Reasoning for Explainable Image Forgery Detection

arXiv:2609.39066v1 Announce Type: new Abstract: Conventional image forgery detection methods produce binary scores or pixel-level masks without interpretable evidence, while recent multimodal large l...

By Zhiya Tan, Jing Huang, Changtao Miao, Lin Tan, Xin Zhang, Weiwei Feng, Jianshu Li, Joey Tianyi Zhou

llms agents reinforcement-learning fine-tuning multimodal benchmarks

arXiv Computer Vision

Oct 1

Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation

arXiv:2609.39120v1 Announce Type: new Abstract: On-policy distillation (OPD) improves reasoning by providing token-level supervision from a teacher on a student's own trajectories. Existing methods p...

By Siyuan Liu, Kanghui Tian, Yue Duan, Yutao He, Shangdong Yang, Jian Zhang, Yinghuan Shi

efficiency multimodal benchmarks

arXiv Computer Vision

Oct 1

Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models

arXiv:2609.39134v1 Announce Type: new Abstract: Visual-token compression improves the efficiency of large vision-language models, but can expose failures that full-token evaluation misses. We study a...

By Shilinlu Yan, Bowen Chen, Yuechen Zhang, Zhenhong Zhou, Li Sun, Sen Su

llms multimodal safety

arXiv Computer Vision

Oct 1

Asking the World: Generalist Physical Reasoning through Agentic World Modeling and Probing

arXiv:2609.39135v1 Announce Type: new Abstract: Physical reasoning from video requires inferring latent physical properties and dynamics beyond direct observation. Direct VLM inference remains unreli...

By Shenxiang Zeng, Chen Yang, Peiyao Chen, Guohui Zhang, Jiansheng Fan, Chen Wang

llms agents multimodal

arXiv Computer Vision

Oct 1

When Can Text Replace Vision? Structural Bottlenecks in Diagram Reasoning

arXiv:2609.39142v1 Announce Type: new Abstract: Can structured text replace vision for diagram reasoning? A wrong answer after textualization can arise because the representation omits information th...

By Yunbei Zhang, Janet Wang, Jihun Hamm, Chandan K Reddy

llms multimodal

arXiv Computer Vision

Oct 1

Uruqi: Learning Spatial Cognition from Visual Experience

arXiv:2609.39195v1 Announce Type: new Abstract: Spatial intelligence requires maintaining a coherent understanding of the world as the embodied agent moves. Like humans, the agent must use its own mo...

By Shichao Li, Meiqi Wang, Fei Su, Zhicheng Zhao

llms agents robotics multimodal benchmarks

Get the daily briefing