Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

8,098 stories · RSS feed

arXiv AI

Sep 15

Bangla Sentence Function Classification: Corpus Development, Model Benchmarking, and Interpretability

arXiv:2609.13869v1 Announce Type: cross Abstract: Automatic sentence function identification is important for many downstream natural language processing (NLP) applications such as dialogue systems,...

By Swapnil Kundu Argha, Abdullah Al Shafi, Rowzatul Zannat, Shoumik Barman Polok, Abdul Muntakim, Jannatul Ferdousi, M. A. Moyeen

nlp multimodal benchmarks safety

arXiv Computer Vision

Sep 15

LG-VLN: A Zero-Shot Vision-and-Language Navigation Framework with LangGraph State Orchestration

arXiv:2609.15098v1 Announce Type: new Abstract: Continuous-environment vision-and-language navigation (VLN-CE) requires interpreting natural-language instructions in unseen 3D environments and execut...

By Jianhe Zhao, Yanhua Qiu, Zhiyu Zhang, Zibo Zhao, Jinhua Xie

agents computer-vision multimodal

arXiv AI

Sep 15

RA-CoA: Training-free Fashion Image Captioning via Retrieval-Augmented Chain-of-Attributes

arXiv:2609.14100v1 Announce Type: cross Abstract: Fashion Image Captioning (FIC) plays a vital role in enhancing user experience and product search in e-commerce platforms. Unlike natural scene image...

By Abhirama Subramanyam Penamakuri, Shreya Shukla, Anand Mishra

llms rag fine-tuning multimodal

arXiv AI

Sep 15

Graph-Transformer Fraud Detection with Self-Supervised Pretraining and Conformal Risk Control

arXiv:2609.14234v1 Announce Type: cross Abstract: Financial fraud in corporate transaction networks has grown more coordinated and harder to detect with rule-based engines and with classical learning...

By Sergei, Komarov

llms multimodal benchmarks safety

arXiv AI

Sep 15

AURA: Unified Multimodal Framework for Conversational Music Editing

arXiv:2609.14344v1 Announce Type: cross Abstract: Instruction-guided music editors typically process each request independently, limiting their ability to support workflows in which users progressive...

By Quoc-Huy Trinh, Minh-Van Nguyen, Debesh Jha

llms multimodal

arXiv AI

Sep 15

A Generative AI Integrated Multimodal Framework for Low-Latency Multi-Camera Person Re-Identification

arXiv:2609.14419v1 Announce Type: cross Abstract: Person re-identification (ReID) is essential for multi-camera surveillance and tracking, yet remains difficult due to viewpoint and illumination chan...

By Leon Fernando, C Dombawala, P. Hettigoda, Vanodhya G. Warnasooriya, Ishara Neranjana, Rashmika Nawaratne

llms rag multimodal benchmarks

arXiv AI

Sep 15

Bridging the Modality Gap in Long-Form Clinical Audio: A Comparative Study of Lightweight and Heavyweight End-to-End SOAP Generation

arXiv:2609.14467v1 Announce Type: cross Abstract: Automating clinical documentation from long-form doctor-patient conversations remains challenging for modern audio-language models. While cascaded AS...

By Ziyu Zhang, Mingchen Shao, Wenjie Tian, Tianlun Zuo, Longhao Li, Lei Xie

llms nlp fine-tuning multimodal safety

arXiv AI

Sep 15

Compositional SVG Generation via VLM-Driven Hierarchical Semantic Parsing

arXiv:2609.14657v1 Announce Type: cross Abstract: While Vision-Language Models (VLMs) excel at visual reasoning, generating structured, editable Scalable Vector Graphics (SVG) remains a fundamental c...

By Sehwan Park, Taehoon Kim, Geonhee Han, Dohyun Kim, Seung Wook Kim, Paul Hongsuck Seo

llms agents multimodal benchmarks

arXiv Computer Vision

Sep 15

GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Driving

arXiv:2609.15169v1 Announce Type: new Abstract: Driving vision-language-action (VLA) models increasingly reason before acting, but their intermediate reasoning is often weakly grounded in physical sc...

By Xiao Liu, Haoyu Li, Jianghao Leng, Lin Wang, Chao Sun

agents reinforcement-learning robotics multimodal benchmarks

arXiv Computer Vision

Sep 15

BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender

arXiv:2609.15478v1 Announce Type: new Abstract: Multimodal agents can create complex videos in software such as Blender by coding without relying on diffusion models. Yet video understanding benchmar...

By Yolo Y. Tang, Daiki Shimada, Jiayue Meng, Jing Bi, Pinxin Liu, Yicheng Wang, Yunzhong Xiao, Zhangyun Tan, Zeliang Zhang, Chao Huang, Susan Liang, Qianxiang Shen, Luchuan Song, Ali Vosoughi, Mingqian Feng, Melika Filvantorkaman, Chenliang Xu

agents diffusion nlp multimodal benchmarks

arXiv Computer Vision

Sep 15

AnchorGUI: Asymmetric Memory for Dual-Scale Learning in GUI Navigation

arXiv:2609.15457v1 Announce Type: new Abstract: Vision-Language Models (VLMs) enable autonomous GUI navigation, but agents still struggle to process and learn from dense, continuous visual histories....

By Shengjie Jin, Zelong Sun, Hengbo Xu, Yanbiao Ma, Zhiwu Lu

llms agents efficiency multimodal benchmarks

arXiv Computer Vision

Sep 15

What a gated sensing pipeline never looks at: bandwidth reduction and the misses behind it

arXiv:2609.15579v1 Announce Type: new Abstract: An airborne sensor on a contested link cannot send video, so the appealing move is to send findings instead and report the ratio between the two. We ev...

By Raghu Venkat, Tricha Anjali

computer-vision multimodal

arXiv Computation and Language

Sep 15

SHIFT-M3: Pre-fusion Alignment-based Consistency Screening for Multimodal ECG Record Integrity

arXiv:2609.13874v1 Announce Type: new Abstract: Multimodal clinical AI typically assumes that the waveform, report, metadata, and downstream predictions attached to a record belong to the same patien...

By Md Ashik Khan, Md Nahid Siddique

llms multimodal benchmarks safety

arXiv AI

Sep 15

IMPACT-VLA: Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies

arXiv:2609.15005v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) policies perform robot manipulation tasks using multimodal inputs such as visual observations, proprioceptive states, an...

By Jinwoong Kim, Sangjin Park

robotics multimodal

arXiv AI

Sep 15

Generate to Explore, Select to Exploit: Aligning LLM-based Headline Generation with Personalized Recommendation

arXiv:2609.15094v1 Announce Type: cross Abstract: In industrial recommendation feeds, presenting a static headline for an item often fails to satisfy the diverse, multimodal interests of the user pop...

By Yi Chen, Rufeng Cheng, Qiang Xie, Tao Li

llms multimodal benchmarks

arXiv AI

Sep 15

Refinement-based Flow Policy Optimization

arXiv:2609.15123v1 Announce Type: cross Abstract: Flow-based policies offer an expressive representation for online reinforcement learning, but conventional flow matching requires samples drawn from...

By Bumgeun Park, Hyukjun Yang, Donghwan Lee

diffusion reinforcement-learning multimodal

arXiv AI

Sep 15

AdaVSkip: Adaptive Visual Token Skipping Across Layers For Efficient MLLMs Inference

arXiv:2609.15131v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) require substantial computation to process numerous visual tokens across all transformer layers. Most method...

By Yuyao Sun, Tao Deng, Shuang Li, Deqing Wang

llms reinforcement-learning multimodal

arXiv Machine Learning

Sep 15

From matrix inversion to constraints: provably tighter confidence regions for importance weights in label shift

arXiv:2609.14802v1 Announce Type: cross Abstract: Importance weights are essential in domain adaptation under label shift, yet their utility is often undermined by the finite sample uncertainty assoc...

By Mushan Li, Kihyun Han, Yanyuan Ma

agents robotics fine-tuning multimodal benchmarks

arXiv AI

Sep 15

Pre-PEFT Probing: Weight Statistics and Perturbation Robustness for Layer Selection in VLM Vision Encoders

arXiv:2609.15229v1 Announce Type: cross Abstract: We propose a pre-fine-tuning probing method for Parameter-Efficient Fine-Tuning (PEFT) layer selection, aiming to obtain more stable and higher gains...

By Qingtao Xia, Jiahua Bao, Siyao Cheng, Jie Liu

llms fine-tuning multimodal benchmarks

arXiv AI

Sep 15

Planning in the Backbone: DiffAdapterVLA for Native Continuous Trajectory Generation with Driving VLMs

arXiv:2609.15322v1 Announce Type: cross Abstract: Pretrained driving vision-language models (VLMs) integrate visual, route, language, and driving context into rich driving priors, yet their represent...

By Changxin Lu, Xiaoliang Meng, Yu Wu, Rui Huang, Honglin Li, Tao Chen, Kaixuan Zhou, Yadong Shao

llms multimodal

Get the daily briefing