Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

4,626 stories · RSS feed

arXiv AI

Jun 6

TempoVLA: Learning Speed-Controllable Vision-Language-Action Policies

arXiv:2606. 06491v1 Announce Type: cross Abstract: Robot manipulation alternates between low-risk transit phases that call for fast execution and high-risk contact stages that demand slow, precise motion.

By Dong Jing, Jingchen Nie, Tianqi Zhang, Jiaqi Liu, Huaxiu Yao, Zhiwu Lu, Mingyu Ding

reinforcement-learning robotics efficiency multimodal

arXiv AI

Jun 6

UNIVID: Unified Vision-Language Model for Video Moderation

arXiv:2606. 05748v1 Announce Type: cross Abstract: Global-scale video moderation faces a dual challenge: the need for fine-grained multi-modal reasoning and the demand for interpretable outputs to support downstream enforcement.

By Kejuan Yang, Yizhuo Zhang, Mingyuan Du, Yue Zhang, Dixin Zheng, Kaili Zhao, Yang Xiao, Hanzhong Liang, Kenan Xiao

llms multimodal safety

arXiv AI

Jun 6

EasyLens: A Training-Free Plug-and-Play Subtle-Lesion Representation Amplifier for Medical Vision-Language Models

arXiv:2606. 06379v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) have shown increasing potential for clinical image interpretation, including lesion detection and report generation.

By Qiwei Zeng, Hao Wang, Jinghao Lin, Shuchang Ye, Yuezhe Yang, Yige Peng, Haoyuan Che, Jinman Kim, Lei Bi

llms rag multimodal safety

arXiv AI

Jun 6

MAviS: A Multimodal Conversational Assistant For Avian Species

arXiv:2603. 07294v2 Announce Type: replace-cross Abstract: Fine-grained understanding and species-specific multimodal question answering are vital for advancing biodiversity conservation and ecological monitoring.

By Yevheniia Kryklyvets, Mohammed Irfan Kurpath, Sahal Shaji Mullappilly, Jinxing Zhou, Fahad Shabzan Khan, Rao Anwer, Salman Khan, Hisham Cholakkal

llms nlp multimodal benchmarks

arXiv AI

Jun 6

Using street view images and visual LLMs to predict heritage values for governance support: Risks, ethics, and policy implications

arXiv:2601. 06056v2 Announce Type: replace-cross Abstract: During 2025 and 2026, the Energy Performance of Buildings Directive is being implemented in the European Union member states, requiring all member states to have National Building Renovation Plans.

By Tim Johansson, Mikael Mangold, Kristina Dabrock, Anna Donarelli, Ingrid Campo-Ruiz

llms multimodal

arXiv AI

Jun 6

Assessing the Geographic Diversity of AI's Platial Representations in Image Generation

arXiv:2606. 05188v1 Announce Type: cross Abstract: (Gen)AI diversity is not merely an ethical issue.

By Zilong Liu, Krzysztof Janowicz, Mina Karimi

llms diffusion multimodal benchmarks safety

arXiv AI

Jun 6

MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models

arXiv:2606. 05177v1 Announce Type: cross Abstract: Existing multimodal safety benchmarks focus solely on visual inputs and cannot assess Omni Large Language Models (LLMs) that process vision, audio, and text.

By Manh Luong, Tamas Abraham, Junae Kim, Amar Kaur, Rollin Omari, Gholamreza Haffari, Trang Vu, Lizhen Qu, Dinh Phung

llms multimodal benchmarks safety

arXiv AI

Jun 6

MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA

arXiv:2606. 05749v1 Announce Type: cross Abstract: Iterative retrieval-reasoning agents have recently shown promise for multimodal long-document question answering.

By Kaifeng Chen, Hongtao Liu, Qiyao Peng, Jian Yang, Yongqiang Liu, Xiaochen Zhang, Qing Yang

agents nlp multimodal

arXiv AI

Jun 6

Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming

arXiv:2606. 05233v1 Announce Type: cross Abstract: Recent computer-using-agent (CUA) red-teaming papers report prompt-injection attack success rates (ASR) of 42-98%, but these headline numbers cluster on retired models and on the most-vulnerable model in each paper's panel.

By Nicholas Saban

llms agents reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 6

RAS: a Reliability Oriented Metric for Automatic Speech Recognition

arXiv:2604. 24278v3 Announce Type: replace-cross Abstract: Automatic speech recognition systems often produce confident yet incorrect transcriptions under noisy or ambiguous conditions, which can be misleading for both users and downstream applications.

By Wenbin Huang, Yuhang Qiu, Bohan Li, Yiwei Guo, Jing Peng, Hankun Wang, Xie Chen, Kai Yu

reinforcement-learning multimodal

arXiv AI

Jun 6

World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis

arXiv:2606. 05979v1 Announce Type: cross Abstract: We propose world-language-action (WLA) models as a new class of embodied foundation models.

By Yi Yang, Zhihong Liu, Siqi Kou, Yiyang Chen, Yanzhe Hu, Jianbo Zhou, Boyuan Zhao, Zhijie Wei, Xiao Xia, Xueqi Li, Pengfei Liu, Zhijie Deng

llms diffusion robotics multimodal benchmarks

arXiv AI

Jun 6

PerceptUI: LLM Agents as Human-Aligned Synthetic Users for UI/UX Evaluation

arXiv:2606. 05697v1 Announce Type: new Abstract: User interface (UI) and user experience (UX) evaluation is central to product development, yet reliable feedback still relies on recruiting human participants or running online A/B tests, making early-stage iteration slow and costly.

By Nicolas Bougie, Xiaotong Ye, Gian Maria Marconi, Narimasa Watanabe

llms agents fine-tuning multimodal

arXiv AI

Jun 6

I Know What You Meme, Even If it Emerged Today: Understanding Evolving Memes through Open-World Knowledge Acquisition

arXiv:2606. 05316v1 Announce Type: new Abstract: Multimodal memes are dynamic and often require up to date background knowledge for interpretation.

By Shanhong Liu, Rui Cao, Pai Chet Ng, De Wen Soh

multimodal benchmarks

arXiv AI

Jun 6

UniVoice: A Unified Model for Speech and Singing Voice Generation

arXiv:2606. 05852v1 Announce Type: cross Abstract: Text-to-speech (TTS) and singing voice synthesis (SVS) both aim to generate human vocal audio from symbolic inputs, but they impose different requirements on the generation process.

By Junjie Zheng, Huixin Xue, Shihong Ren, Chaofan Ding, Hao Liu, Zihao Chen

llms diffusion multimodal safety

arXiv AI

Jun 6

WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation

arXiv:2606. 06147v1 Announce Type: new Abstract: End-to-end Vision-Language-Action (VLA) models have shown promise in UAV navigation.

By Shengtao Zheng, Kai Li, Weichen Zhang, Yu Meng, Chen Gao, Xinlei Chen, Yong Li, Xiao-Ping Zhang

agents diffusion robotics multimodal benchmarks

arXiv AI

Jun 6

LongSpace: Exploring Long-Horizon Spatial Memory from Perception to Recall in Video

arXiv:2606. 05677v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have advanced image and video understanding and can increasingly handle longer visual inputs.

By Shiqiang Lang, Jing Liu, Haoyang He, Peiwen Sun, Yuanteng Chen, Tao Liu, Lan Yang, Longteng Guo, Honggang Zhang

llms agents robotics multimodal benchmarks

arXiv AI

Jun 6

Where Should Knowledge Enter? A Layered Framework for Knowledge Infusion in Multimodal Iterative Generative Mo

arXiv:2606. 06356v1 Announce Type: new Abstract: Multimodal generative models produce fluent outputs but remain unreliable when generation must respect structured, domain-specific, or safety-critical knowledge.

By Renjith Prasad, Chathurangi Shyalika, Anushka Pawar, Amit Sheth

diffusion fine-tuning multimodal safety

arXiv AI

Jun 6

The Mirage of Performance Gains: Why Contrastive Decoding Fails to Mitigate Object Hallucinations in MLLMs?

arXiv:2504. 10020v4 Announce Type: replace-cross Abstract: Contrastive decoding strategies are widely used to reduce object hallucinations in multimodal large language models (MLLMs).

By Hao Yin, Guangzong Si, Zilei Wang

llms multimodal benchmarks safety

arXiv AI

Jun 6

FUSAR-GPT : A Spatiotemporal Feature-Embedded and Two-Stage Decoupled Visual Language Model for SAR Imagery

arXiv:2602. 19190v4 Announce Type: replace-cross Abstract: Research on the intelligent interpretation of all-weather, all-time Synthetic Aperture Radar (SAR) is crucial for advancing remote sensing applications.

By Xiaokun Zhang, Yi Yang, Ziqi Ye, Baiyun, Xiaorong Guo, Qingchen Fang, Ruyi Zhang, Xinpeng Zhou, Haipeng Wang

llms rag fine-tuning multimodal benchmarks

arXiv AI

Jun 6

AgenticRL: Self-Refining Agentic Reinforcement Learning for Vision-Conditioned UAV Navigation

arXiv:2606. 03963v2 Announce Type: replace-cross Abstract: Deep reinforcement learning has shown strong potential for enabling autonomous robots to learn complex navigational tasks.

By Roohan Ahmed Khan, Yasheerah Yaqoot, Muhammad Ahsan Mustafa, Dzmitry Tsetserukou

llms agents reinforcement-learning robotics fine-tuning multimodal

Get the daily briefing