Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

8,972 stories · RSS feed

arXiv AI

Jul 2

Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection

arXiv:2606. 25375v2 Announce Type: replace-cross Abstract: With the rapid adoption of generative AI, synthetic medical images pose growing risks, including diagnostic deception and insurance fraud.

By Ching-Hao Chiu, Hao-Wei Chung, Gelei Xu, Xueyang Li, Pin-Yu Chen, John Kheir, Meysam Ghaffari, Carlos Morato, Ahmed Abbasi, Yiyu Shi

llms multimodal benchmarks

arXiv AI

Jul 2

What's Hidden Matters: Identifying Planning-Critical Occluded Agents using Vision-Language Models

arXiv:2607. 00283v1 Announce Type: cross Abstract: Autonomous vehicles must safely navigate complex environments where planning-critical agents may be hidden from view.

By Amirhosein Chahe, Tyler Naes, Jovin D'sa, Faizan M. Tariq, Sangjae Bae, Lifeng Zhou, David Isele

llms agents robotics fine-tuning multimodal benchmarks

arXiv AI

Jul 2

SleepLM: Natural-Language Intelligence for Human Sleep

arXiv:2602. 23605v2 Announce Type: replace Abstract: We present SleepLM, a family of sleep-language foundation models that enable human sleep alignment, interpretation, and interaction with natural language.

By Zongzhe Xu, Zitao Shuai, Eideen Mozaffari, Ravi S. Aysola, Rajesh Kumar, Yuzhe Yang

multimodal benchmarks safety

arXiv Machine Learning

Jul 2

MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation

arXiv:2602. 21397v2 Announce Type: replace-cross Abstract: Prompt learning has become a dominant paradigm for adapting vision-language models (VLMs) such as CLIP to downstream tasks without modifying pretrained weights.

By Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh, Ramtin Pedarsani

llms rag multimodal benchmarks safety

arXiv Machine Learning

Jul 2

AdaBoosting Text Prompts for Vision-Language Models

arXiv:2607. 00684v1 Announce Type: new Abstract: The classification accuracy of pretrained Vision-Language Models (VLMs) relies on the quality of the text prompts.

By Seokhee Jin, Changhwan Sung, Sunung Mun, Hoyoung Kim, Jungseul Ok

llms multimodal benchmarks

arXiv AI

Jul 2

Interact3D: Compositional 3D Generation of Interactive Objects

arXiv:2603. 16085v2 Announce Type: replace-cross Abstract: Recent breakthroughs in 3D generation have enabled the synthesis of high-fidelity individual assets.

By Hui Shan, Keyang Luo, Ming Li, Sizhe Zheng, Yanwei Fu, Zhen Chen, Xiangru Huang

llms agents multimodal safety

arXiv Machine Learning

Jul 2

MindAU: EEG-Conditioned Facial Action Unit Editing via Dual-Stream Manifold Alignment

arXiv:2607. 00410v1 Announce Type: cross Abstract: Recent brain decoding studies have made substantial progress in reconstructing externally perceived visual content from neural signals.

By Zhenhang Li, Xin Zhou, Hao Deng, Lijun Yin

rag diffusion multimodal benchmarks safety

arXiv Machine Learning

Jul 2

Multiplicity is an Inevitable and Inherent Challenge in Multimodal Learning

arXiv:2505. 19614v2 Announce Type: replace Abstract: Multimodal learning has seen remarkable progress, particularly with large-scale pre-training across various modalities.

By Sanghyuk Chun, Olga Russakovsky

multimodal benchmarks safety

arXiv Machine Learning

Jul 2

IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Video

arXiv:2603. 16432v3 Announce Type: replace-cross Abstract: Unsupervised physical parameter estimation from video lacks a common benchmark: existing methods evaluate on non-overlapping synthetic data, the sole real-world dataset is restricted to single-body systems, and no established protocol addresses governing-equation identification.

By Rasul Khanbayov, Mohamed Rayan Barhdadi, Erchin Serpedin, Hasan Kurban

llms multimodal benchmarks

arXiv AI

Jul 2

FLAT: Revealing Hidden Latent-Conditioned Backdoor Failures in Federated Learning

arXiv:2508. 04064v2 Announce Type: replace-cross Abstract: Horizontal federated learning (HFL) backdoor audits often summarize model behavior through clean accuracy (CA), mean attack success rate (ASR), or a single known-trigger test.

By Tuan Nguyen, Sze Jue Yang, Khoa D. Doan, Chee Seng Chan, Kok-Seng Wong

multimodal

arXiv Machine Learning

Jul 2

VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning

arXiv:2511. 17731v2 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) prompting has proven remarkably effective for eliciting complex reasoning in large language models (LLMs).

By Lingxiao Li, Yifan Wang, Xinyan Gao, Chen Tang, Xiangyu Yue, Chenyu You

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jul 2

Rule-VLN: Bridging Perception and Compliance via Semantic Reasoning and Geometric Rectification

arXiv:2604. 16993v2 Announce Type: replace Abstract: As embodied AI transitions to real-world deployment, the success of the Vision-and-Language Navigation (VLN) task tends to evolve from mere reachability to social compliance.

By Jiawen Wen, Penglei Sun, Wenjie Zhang, Suixuan Qiu, Weisheng Xu, Xiaofei Yang, Xiaowen Chu

agents robotics multimodal benchmarks safety

arXiv AI

Jul 2

GaussianFusion: Unified 3D Gaussian Representation for Multi-Modal Fusion Perception

arXiv:2607. 00746v1 Announce Type: cross Abstract: The bird's-eye view (BEV) representation enables multi-sensor features to be fused within a unified space, serving as the primary approach for achieving comprehensive 3D perception.

By Xiao Zhao, Chang Liu, Mingxu Zhu, Zheyuan Zhang, Linna Song, Qingliang Luo, Chufan Guo, Kuifeng Su

computer-vision multimodal safety

arXiv AI

Jul 2

GameDevBench: Evaluating Agentic Capabilities Through Game Development

arXiv:2602. 11103v2 Announce Type: replace Abstract: Despite rapid progress on coding agents, progress on their multimodal counterparts has lagged behind.

By Wayne Chi, Yixiong Fang, Arnav Yayavaram, Siddharth Yayavaram, Seth Karten, Qiuhong Anna Wei, Runkun Chen, Alexander Wang, Valerie Chen, Ameet Talwalkar, Chris Donahue

llms agents multimodal benchmarks

arXiv AI

Jul 2

Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks

arXiv:2603. 11689v3 Announce Type: replace Abstract: Frontier Multimodal Large Language Models (MLLMs) exhibit remarkable capabilities in Visual-Language Comprehension (VLC) tasks.

By Mei Chee Leong, Ying Gu, Hui Li Tan, Liyuan Li, Nancy Chen

llms multimodal benchmarks

arXiv AI

Jul 2

LongVQUBench: Benchmarking Long-Term Video Quality Understanding of Vision-Language Models

arXiv:2607. 01086v1 Announce Type: cross Abstract: The evaluation of long-term video quality understanding remains an open challenge for large vision-language models (LVLMs).

By Arpita Nema, Hanwei Zhu, Xi Zhang, Weisi Lin

llms multimodal benchmarks

arXiv AI

Jul 2

XSkill: Continual Learning from Experience and Skills in Multimodal Agents

arXiv:2603. 12056v3 Announce Type: replace Abstract: Multimodal agents can now tackle complex reasoning tasks with diverse tools, yet they still suffer from inefficient tool use and inflexible orchestration in open-ended settings.

By Guanyu Jiang, Zhaochen Su, Xiaoye Qu, Yi R. Fung

agents nlp multimodal benchmarks

arXiv AI

Jul 2

FurnitureVLA: Learning Long-Horizon Bimanual Furniture Assembly with Vision-Language-Action Model

arXiv:2607. 01212v1 Announce Type: cross Abstract: Current work on robot furniture assembly mostly focuses on toy-scale settings or single-arm manipulation.

By Chenyang Ma, Yue Yang, Radu Corcodel, Siddarth Jain, Andrew Wu, Chiori Hori, Diego Romeres

robotics multimodal

Hugging Face Trending Papers

Jul 2

SINA: A Fully Automated Circuit Schematic Image to Netlist Generator Using Artificial Intelligence

Recent advances in Artificial Intelligence (AI) have revolutionized Electronic Design Automation (EDA), particularly through Large Language Models (LLMs) for circuit design tasks. However, their application to analog and mixed-signal domains remains limited by the lack of machine-readable representations of existing circuit design knowledge.

llms computer-vision multimodal benchmarks

Hugging Face Trending Papers

Jul 1

FurnitureVLA: Learning Long-Horizon Bimanual Furniture Assembly with Vision-Language-Action Model

Current work on robot furniture assembly mostly focuses on toy-scale settings or single-arm manipulation. We introduce FurnitureVLA, the first systematic study of real-scale bimanual furniture assembly using Vision-Language-Action models (VLAs).

robotics multimodal

Get the daily briefing