Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

7,971 stories · RSS feed

arXiv Computer Vision

Sep 30

GlassFormer: Learning Real-time Glass Segmentation using Radar-Depth Fusion

arXiv:2609.36844v1 Announce Type: new Abstract: Transparent surfaces are ubiquitous in built environments, yet they remain a persistent failure case for robotic perception. RGB cameras perceive the b...

By Suhani Grover, Astik Srivastava, Viswas Dinesh, Avinash Sharma, K. Madhava Krishna

llms computer-vision robotics multimodal

arXiv Computer Vision

Sep 30

SFE-VGGT: Source-Free VGGT Distillation for Event-Based Monocular Depth Estimation

arXiv:2609.36929v1 Announce Type: new Abstract: Recent event-based depth estimation methods successfully transfer geometric priors from vision foundation models via cross-modal distillation. However,...

By Thai Duy Nguyen, Addison Lin Wang

computer-vision efficiency multimodal

arXiv Computer Vision

Sep 30

Representation Dynamics Reveal Semantic Saliency and Similarity for Visual Token Pruning in MLLMs

arXiv:2609.36916v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) incur high inference latency from long visual token sequences. Existing pruning methods commonly use attention...

By Weixuan Li, Zikun Zhou, Xinyi Zhuang, Xinyan Guo, Rui Tian, Chuyao Zhang, Lin Gao

llms efficiency multimodal

arXiv Computer Vision

Sep 30

VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimation

arXiv:2609.37003v1 Announce Type: new Abstract: Vessel perception from space is crucial for a wide range of maritime applications, from traffic monitoring to environmental protection. However, most e...

By Danfeng Hong, Chenyu Li, Jocelyn Chanussot

computer-vision multimodal benchmarks

arXiv Computer Vision

Sep 30

OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models

arXiv:2609.37052v1 Announce Type: new Abstract: Omnimodal large language models (Omni-LLMs) encode audio and visual streams into temporally interleaved token sequences for multimodal reasoning. Howev...

By Yuchen Deng, Zidang Cai, Feidiao Yang, Yufei Wang, Jie Wang, Hai-Tao Zheng, Yuxing Han

llms multimodal benchmarks

arXiv Computer Vision

Sep 30

Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack

arXiv:2609.37096v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) consistently struggle with fine-grained visual counting, yet the underlying causes remain poorly understood. I...

By Liwei Che, Yihao Quan, Sen Fang, Hongyi Wang, Ranjay Krishna, Ruixiang Tang, Vladimir Pavlovic

llms fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 30

SAM Meets VLM: Parameter-Decoupled Full-Parameter Training for Unified Medical Reasoning and Segmentation

arXiv:2609.37283v1 Announce Type: new Abstract: Medical multimodal large language models (MLLMs) are increasingly expected not only to answer clinical questions, but also to localize the visual evide...

By Xuyang Cao, Enyou Liu, Jun Zhao, Zhuoyun Liu, Jintao Fei, Leo

llms computer-vision fine-tuning multimodal benchmarks safety

arXiv Computer Vision

Sep 30

What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation

arXiv:2609.37317v1 Announce Type: new Abstract: Omnimodal evaluation should go beyond independent text, image, and speech production: individually plausible outputs may not express a coherent shared...

By Sieun Hyeon, Yejoon Lee, Mintaek Lim, Woojin Kim, Jaeik Kim, Jaeyoung Do

llms agents diffusion multimodal benchmarks

arXiv Computer Vision

Sep 30

When to Retrieve, When to Stay: Uncertainty-Aware Temporal Evidence Allocation for Streaming Video-LLMs

arXiv:2609.37345v1 Announce Type: new Abstract: Streaming video understanding requires Video Large Language Models (Video-LLMs) to reason over continuous visual streams under causal constraints. As t...

By Xiang Hu, Jiazuo Yu, Lu Zhang, Yunzhi Zhuge, Huchuan Lu

llms multimodal benchmarks

arXiv Computer Vision

Sep 30

MG-Thinker: Bi-Axial Self-Reflection for Multi-Image Reasoning Grounding

arXiv:2609.37374v1 Announce Type: new Abstract: Reinforcement learning (RL) has recently delivered substantial gains in multimodal reasoning, opening a promising route for fine-grained visual percept...

By Heyu Huang, Chi Chen, Zonghao Guo, Yuhua Li, Maosong Sun, Ruixuan Li

reinforcement-learning multimodal benchmarks

arXiv Computer Vision

Sep 30

Evaluating the Evaluators: Diagnosing Large Multimodal Models for AI-Generated Image Assessment

arXiv:2609.37576v1 Announce Type: new Abstract: With the rapid advancement of text-to-image (T2I) generation, robust evaluation becomes critical yet challenging, as traditional metrics fail to captur...

By Yu Zhao, Jiarui Wang, Huiyu Duan, Ye Zhao, Jutao Tang, Juntong Wang, Guangtao Zhai, Xiongkuo Min

llms diffusion multimodal benchmarks safety

arXiv Computer Vision

Sep 30

Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding

arXiv:2609.37655v1 Announce Type: new Abstract: Advancing spatial intelligence in Multimodal Large Language Models (MLLMs) is bottlenecked by the scarcity of complex, scalable 3D question-answer (QA)...

By Jiayu Ying, Qijian Tian, Ruijie Xu, Xinnan Zhu, Daoguo Dong, Jiachen Xu, Xin Tan

llms agents nlp robotics fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 30

Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning

arXiv:2609.37656v1 Announce Type: new Abstract: Large vision-language models (LVLMs) exhibit strong reasoning capabilities, yet the visual and textual evidence supporting the generated responses rema...

By Bowen Yuan, Danny Wang, Ruihong Qiu, Zijian Wang, Zi Huang

llms multimodal benchmarks

arXiv Computer Vision

Sep 30

EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory

arXiv:2609.37923v1 Announce Type: new Abstract: Agents can learn from past executions, but enabling different agents to reuse and build on one another's experience remains challenging. We introduce E...

By Ziyun Zeng, Hang Hua, Shaden Alshammari, Rogerio Feris, William T. Freeman, Jiebo Luo

agents multimodal benchmarks

arXiv Computer Vision

Sep 30

EVO-WAM: Evolving World Action Models through Video-Action Verification

arXiv:2609.38057v1 Announce Type: new Abstract: Improving robot policies on new tasks without collecting additional expert demonstrations remains a central challenge in robot learning. World action m...

By Shiyang Zhou, Xionghao Wu, Wenbo Li, Shenghe Zheng, Jiyao Zhang, Songsong Yu, Yijun Yang, Jianhui Liu, Haoze Sun, Senqiao Yang, Li Jiang, Jingyong Su, Haoyang Huang, Zhuotao Tian

llms robotics multimodal

arXiv Computer Vision

Sep 30

HelixWorld: A Real-time Interactive Audio-Visual World Model

arXiv:2609.38123v1 Announce Type: new Abstract: World simulation is inherently multisensory, demanding synchronized visual and acoustic dynamics in real time. Yet prevailing interactive world models...

By Lei Ke, Jiahao Pan, Zeyue Tian, Jiaming Wang, Haoyuan Huang, Kam Man Wu, Pengjun Fang, Hongyu Liu, Chenyang Qi, Lin Wang, Ruibin Yuan, Weijia Chen, Fangneng Zhan, Qifeng Chen, Wei Xue, Yike Guo

efficiency multimodal benchmarks

arXiv Computer Vision

Sep 30

VideoLoop: Looped Working Memory Against Semantic Thrashing in Long-Form Video Agents

arXiv:2609.38119v1 Announce Type: new Abstract: Long-form video understanding requires multimodal agents to iteratively gather evidence over many reasoning steps. However, most existing agentic metho...

By Jinfa Huang, Jianming Xu, Jingyang Lin, Zhengyuan Yang, Jiebo Luo

llms agents multimodal

arXiv Computer Vision

Sep 30

Selective Channel Restoration for Backdoored Vision-Language Models

arXiv:2609.37759v1 Announce Type: cross Abstract: Vision-language models (VLMs) exhibit strong multimodal capabilities but remain vulnerable to backdoors implanted through poisoned fine-tuning data....

By Shuming Liu, Zhifang Zhang, Suqin Yuan, Khin Mi Mi Aung, Zhuoyi Lin, Lei Feng

llms fine-tuning multimodal

arXiv Computer Vision

Sep 30

A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding

arXiv:2603.14733v2 Announce Type: replace Abstract: Multimodal Large Language Models have achieved strong performance in single-video understanding, yet their ability to reason across multiple videos...

By Yue Zhang, Liqiang Jing, Jia Li, Yapeng Tian, Xinya Du, Yunhui Guo, Vibhav Gogate

llms agents computer-vision multimodal benchmarks

arXiv Computer Vision

Sep 30

UHR-Micro: Diagnosing and Mitigating the Resolution Illusion in Earth Observation VLMs

arXiv:2605.12237v2 Announce Type: replace Abstract: Vision-Language Models (VLMs) are increasingly used to analyze ultra-high-resolution (UHR) Earth observation imagery, yet they face a severe scale...

By Shuo Ni, Tong Wang, Jing Zhang, Di Wang, He Chen, Haonan Guo, Ning Zhang, Bo Du

llms multimodal benchmarks

Get the daily briefing