Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

8,954 stories · RSS feed

arXiv Machine Learning

Jul 22

Subject-Conditioned Glucose Forecasting in Type-1 Diabetes

arXiv:2607. 19006v1 Announce Type: new Abstract: Accurate forecasting of blood glucose concentration is key in the management of Type 1 Diabetes, facilitating early detection of adverse glycemic events and supporting timely therapeutic interventions.

By Giorgia Rigamonti, Mirko Paolo Barbato, Davide Marelli, Paolo Napoletano

multimodal benchmarks

arXiv AI

Jul 22

Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusions

arXiv:2607. 19061v1 Announce Type: cross Abstract: Hateful optical illusions expose a serious gap in current multimodal safety systems.

By Qianpu Chen, Derya Soydaner

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jul 22

Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges

arXiv:2607. 19011v1 Announce Type: cross Abstract: Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, shared cultural knowledge, and communicative intent rather than literal scene description.

By Tuo Liang, Zhe Hu, Disheng Liu, Jing Li, Yu Yin

llms multimodal benchmarks safety

arXiv AI

Jul 22

AnchorRefine: Synergy-Manipulation Based on Trajectory Anchor and Residual Refinement for Vision-Language-Action Models

arXiv:2604. 17787v2 Announce Type: replace-cross Abstract: Precision-critical manipulation requires both global trajectory organization and local execution correction, yet most vision-language-action (VLA) policies generate actions within a single unified space.

By Tingzheng Jia, Kan Guo, Lanping Qian, Yongli Hu, Daxin Tian, Guixian Qu, Chunmian Lin, Baocai Yin, Jiapu Wang

diffusion robotics multimodal

arXiv AI

Jul 22

FormGym: Doing Paperwork with Agents

arXiv:2506. 14079v4 Announce Type: replace Abstract: Completing paperwork is a challenging and time-consuming problem.

By Matthew Toles, Rattandeep Singh, Isaac Song, Zhou Yu

llms agents computer-vision multimodal benchmarks

arXiv AI

Jul 22

Why Do Vision Language Models Struggle To Recognize Human Emotions?

arXiv:2604. 15280v2 Announce Type: replace-cross Abstract: Understanding emotions is a fundamental ability for intelligent systems to be able to interact with humans.

By Madhav Agarwal, Sotirios A. Tsaftaris, Laura Sevilla-Lara, Steven McDonagh

llms multimodal safety

arXiv Machine Learning

Jul 22

Benchmarking Deep Learning Approaches for AEC Engineering Drawing Layout Detection and Information Extraction

arXiv:2607. 18997v1 Announce Type: cross Abstract: Information Extraction (IE) from Architecture, Engineering, and Construction (AEC) drawings remains hindered by manual inefficiency, while Layout Detection, a vital 'middleware' organizing graphical and textual hierarchies, is underexplored.

By Tianyang Huang, Alessio Lombardi, Ahmed Elnagar, Ahmed Zalouk, George Paul, Sepehr Najjarpour, Arvid Sigurdsson, Khalid Ismail, Mohamed Ragab, Edlira Vakaj

llms multimodal benchmarks

arXiv AI

Jul 22

MAGE: Human-Like Macro Placement via Agentic Multimodal Reasoning

arXiv:2607. 18536v1 Announce Type: new Abstract: Macro placement still requires substantial manual refinement in industrial physical design flows.

By Andrew B. Kahng, Sayak Kundu, Bodhisatta Pramanik

agents multimodal safety

arXiv AI

Jul 22

GUIDED Network-Agnostic Feature Initialization for Spatial Transferability in GNN-based Models

arXiv:2607. 19270v1 Announce Type: cross Abstract: The Traffic Assignment Problem is a fundamental but computationally expensive component of transportation planning.

By Alessandro Scalese, Santhanakrishnan Narayanan, Constantinos Antoniou

rag fine-tuning multimodal benchmarks

arXiv AI

Jul 22

Appearance Pointers -- Multimodal Region Control of Diffusion Transformers

arXiv:2607. 19344v1 Announce Type: cross Abstract: Controllable image generation remains challenging for creative professionals, who often require precise regional control over materials, object identities, and spatial arrangements that cannot be reliably achieved through text prompting alone.

By Rahul Sajnani, Yulia Gryaditskaya, Radom\'ir M\v{e}ch, Srinath Sridhar, Matheus Gadelha

llms diffusion multimodal

arXiv Machine Learning

Jul 22

Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning

arXiv:2607. 18615v1 Announce Type: cross Abstract: Machine unlearning for vision-language models (VLMs) remains underexplored.

By Zijie Liu, Jinhao Duan, Gaowen Liu, Sijia Liu, Tianlong Chen

llms multimodal

arXiv AI

Jul 22

Attributes Should Come from Images, Not Class Names: Distribution-Conditioned Attribute Selection for Vision-Language Models

arXiv:2607. 18695v1 Announce Type: cross Abstract: A popular route to interpretable zero-shot classification asks a large language model (LLM) to describe each class name and prompts CLIP with the resulting descriptors.

By Gautam Rajendrakumar Gare, Jia Shi, Zhiqiu Lin, Deepak Pathak, John Galeotti, Deva Ramanan

llms rag multimodal

arXiv AI

Jul 22

Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing

arXiv:2607. 19064v1 Announce Type: cross Abstract: Large-scale visual generators are increasingly capable but costly to train, fine-tune, and deploy.

By Xinjie Zhang, Peng Zhang, Shicheng Zheng, Jinghao Guo, Zhaoyang Jia, Yifei Shen, Xun Guo, Yuxuan Luo, Jiahao Li, Wenxuan Xie, Fanyi Pu, Xiaoyi Zhang, Kaichen Zhang, Zongyu Guo, Tianci Bi, Dongnan Gui, Zhening Liu, Zimo Wen, Zihan Zheng, Senqiao Yang, Xiao Li, Jinglu Wang, Bin Li, Yan Lu

llms diffusion nlp reinforcement-learning fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jul 22

PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image

arXiv:2607. 19261v1 Announce Type: cross Abstract: Whole-slide image (WSI) diagnosis requires identifying diagnostically relevant regions, examining them across magnifications, and integrating multi-scale evidence.

By Dankai Liao, Tianyi Zhang, Yufeng Wu, Xinyue Zhang, Qiaochu Xue, Zeyu Liu, Dachun Zhao, Linghan Cai, Yueming Jin

llms agents diffusion multimodal benchmarks

arXiv AI

Jul 22

Querying Multimodal Scientific Papers with AI: Practices and Preferences Across Blind, Low-Vision, and Sighted Scientists

arXiv:2607. 18514v1 Announce Type: cross Abstract: Visual diagrams, figures, and tables are central to scientific papers, and convey information beyond what is captured in text.

By Arnavi Chheda-Kothary, Lucy Lu Wang, Joseph Chee Chang, Jonathan Bragg

llms multimodal

arXiv AI

Jul 22

When Visual Evidence is Ambiguous: Pareidolia as a Diagnostic Probe for Vision Models

arXiv:2603. 03989v2 Announce Type: replace-cross Abstract: When visual evidence is ambiguous, vision models must decide how to interpret face-like patterns.

By Qianpu Chen, Derya Soydaner, Rob Saunders

llms computer-vision multimodal safety

arXiv AI

Jul 22

Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigation

arXiv:2602. 15875v2 Announce Type: replace-cross Abstract: Current Visual-Language Navigation (VLN) methodologies face a trade-off between semantic understanding and control precision.

By Zhenxing Xu, Yihong Lu, Weidong Bao, Zhengqiu Zhu, Jingxuan Zhou, Zhichuang Wang, Ji Wang, Lihua Liu, Wei He

llms multimodal benchmarks

arXiv AI

Jul 22

NaviAIS: A Scenario-Level Vessel Trajectory Prediction Dataset withVectorized Lane Priors and the NaviLane Forecasting Framework

arXiv:2607. 18887v1 Announce Type: new Abstract: Vessel trajectory prediction in complex maritime environments is essential for traffic management, collision warning, route planning, and autonomous navigation.

By Yuan Gui, Hongchen Luo, Liqi Qu, Longyue Fu, Jiao Wang

agents multimodal

arXiv AI

Jul 22

Quality Action Assurance: Multimodal Verification of Examiner Claims in VR OSCEs

arXiv:2607. 19063v1 Announce Type: new Abstract: Objective Structured Clinical Examinations (OSCEs) are the gold standard for assessing clinical competence, yet scoring remains vulnerable to examiner subjectivity, fatigue, and cognitive bias.

By Harry Rogers, Sally Shiels, Ashley Tomlinson, James Thomas, James Aylward, Nathan Gauge, Helen Higham, Alison Noble

llms multimodal safety

arXiv AI

Jul 22

Breaking the Homogeneity Assumption: Specialized Multi-Generator Adversarial Learning for Rare Failure Detection in Predictive Maintenance

arXiv:2607. 19153v1 Announce Type: cross Abstract: Supervised learning models in the predictive maintenance field are regularly trained on highly imbalanced industrial datasets: machine failures occur rarely but have a disproportionate effect on operations.

By Alexis Lazanas, Georgios Kampouropoulos

diffusion multimodal safety

Get the daily briefing