Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

4,022 stories · RSS feed

arXiv AI

Jul 23

Leveraging Offline Supervision for Efficient and Generalizable Reinforcement Learning in Large-Scale Vision-Language-Action Models

arXiv:2607. 19399v1 Announce Type: cross Abstract: It is commonly observed that online reinforcement learning (RL) produces better-performing strategies than offline methods across a broad range of performance measures.

By Dmitriy Poyarkov, Aleksei Staroverov, Aleksandr I. Panov

reinforcement-learning fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 23

Deep Shape Regression for Planar Curves with Multimodal Covariates

arXiv:2607. 19600v1 Announce Type: cross Abstract: The shape of a planar curve is the geometric information that remains once translation, rotation, scale and reparametrisation are removed and is of interest in many health applications, e.

By Manuel Pfeuffer, Roshan Prakash Rane, Hadya Yassin, Kerstin Ritter, Sonja Greven

multimodal safety

arXiv Machine Learning

Jul 23

A Deep Learning Framework for Predicting Solar EUV Irradiance During Significant Flares

arXiv:2607. 19597v1 Announce Type: cross Abstract: We present FlareEUV, a multimodal deep learning framework for predicting daily extreme ultraviolet (EUV) irradiance at 6.

By Sathvik Soman, Jason T. L. Wang, Haimin Wang, Haodi Jiang

multimodal

arXiv AI

Jul 23

Closing the Lab-to-Store Gap: A Data-Efficient Post-Training and Experience-Driven Learning VLA Framework for Retail Humanoids

arXiv:2607. 20345v1 Announce Type: cross Abstract: Closing the gap between benchmark performance and reliable real-world operation remains a central challenge for Vision-Language-Action (VLA) humanoid robots, which must handle execution errors, distribution shifts, and environmental variability.

By Roger Sala Sis\'o, Tiago Silv\'erio, Jakob Sand, Tran Nguyen Le

robotics fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jul 23

Air Quality Arena: A Large-Scale Multi-Region Ground Monitoring Dataset and Benchmark for Air Quality Forecasting with Time-Series Foundation Models

arXiv:2607. 19381v1 Announce Type: new Abstract: Air pollution causes an estimated 7.

By Rishi Bharadwaj, Manik Gupta, Pandarasamy Arjunan

multimodal benchmarks

arXiv Machine Learning

Jul 23

Don't Fool Me Twice: Adapting to Adversity in the Wild with Experience-Driven Reasoning

arXiv:2605. 31119v2 Announce Type: replace-cross Abstract: In robotics, dangers and adversity modes are often embodiment-specific and relative to each agent.

By Navin Sriram Ravie, Andrew Jong, Krrish Jain, John Liu, Omar Alama, Bijo Sebastian, Sebastian Scherer

llms agents robotics multimodal

arXiv Machine Learning

Jul 23

Missing-by-Design: Certifiable Modality Deletion for Revocable Multimodal Sentiment Analysis

arXiv:2602. 16144v4 Announce Type: replace-cross Abstract: As multimodal systems increasingly process sensitive personal data, the ability to selectively revoke specific data modalities has become a critical requirement for privacy compliance and user autonomy.

By Rong Fu, Ziming Wang, Chunlei Meng, Jiekai Wu, Kangan Qian, Hao Zhang, Simon Fong

rag nlp multimodal benchmarks safety

arXiv AI

Jul 23

Leveraging ChatGPT's Multimodal Vision Capabilities to Rank Satellite Images by Poverty Level: Advancing Tools for Social Science Research

arXiv:2501. 14546v2 Announce Type: replace-cross Abstract: This paper investigates the novel application of Large Language Models (LLMs) with vision capabilities to analyze satellite imagery for village-level poverty prediction.

By Hamid Sarmadi, Ola Hall, Thorsteinn R\"ognvaldsson, Mattias Ohlsson

llms multimodal

arXiv AI

Jul 23

Learning the Arabic Dialect Continuum as a Continuous Space: A Regression Approach to Speaker Origin Prediction

arXiv:2607. 19751v1 Announce Type: cross Abstract: We present a regression-based approach to Arabic dialect geolocation that models dialectal variation as a continuous geographic space rather than discrete categories.

By Mohamed Aziz Khadraoui, Adel Ammar, Bilel Benjdira, Zahid Khan, Skander Turki, Wadii Boulila

llms diffusion multimodal

arXiv AI

Jul 23

G-MAD: A Game-Based Data Generation Framework for Multi-View RGB-T Aerial Object Detection

arXiv:2607. 19942v1 Announce Type: cross Abstract: This work introduces G-MAD, an open-source framework that uses Arma3 to generate synchronized multi-view RGB-T data for aerial object detection.

By Yechan Kim, JongHyun Park, Dongho Yoon, Namhoon Jung, Moongu Jeon

computer-vision multimodal benchmarks safety

arXiv Machine Learning

Jul 23

Diffusion-based Annealed Boltzmann Generators : benefits, pitfalls and hopes

arXiv:2601. 21026v2 Announce Type: replace-cross Abstract: Sampling configurations at thermodynamic equilibrium is a central challenge in statistical physics.

By Louis Grenioux, Maxence Noble

diffusion multimodal

arXiv AI

Jul 23

SynSur: An end-to-end generative pipeline for synthetic industrial surface defect generation and detection

arXiv:2604. 26633v2 Announce Type: replace-cross Abstract: Industrial surface defect inspection suffers from a fundamental data bottleneck: defects are rare, annotations require expert knowledge, and collecting balanced training sets is slow and costly.

By Paul Julius K\"uhn, Mika Pommeranz, Arjan Kuijper, Saptarshi Neil Sinha

diffusion fine-tuning multimodal

arXiv AI

Jul 23

SubQuad: Near-Quadratic-Free Structure Inference with Distribution-Balanced Objectives in Adaptive Receptor framework

arXiv:2602. 17330v5 Announce Type: replace-cross Abstract: Comparative analysis of adaptive immune repertoires at population scale is hampered by two practical bottlenecks: the near-quadratic cost of pairwise affinity evaluations and dataset imbalances that obscure clinically important minority clonotypes.

By Rong Fu, Zijian Zhang, Kun Liu, Jiekai Wu, Xianda Li, Simon Fong

rag efficiency multimodal safety

arXiv AI

Jul 23

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

arXiv:2607. 19894v1 Announce Type: cross Abstract: Large language models (LLMs) are vulnerable to backdoor attacks, where hidden triggers induce malicious outputs.

By Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, Haoyu Wang

llms fine-tuning efficiency multimodal benchmarks

arXiv AI

Jul 23

Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection

arXiv:2512. 16300v3 Announce Type: replace Abstract: Existing image forgery detection (IFD) methods either exploit low-level, semantics-agnostic artifacts or rely on multimodal large language models (MLLMs) with high-level semantic knowledge.

By Fanrui Zhang, Qiang Zhang, Sizhuo Zhou, Jianwen Sun, Chuanhao Li, Jiaxin Ai, Yukang Feng, Yujie Zhang, Wenjie Li, Zizhen Li, Yifan Chang, Jiawei Liu, Kaipeng Zhang

llms agents fine-tuning multimodal

arXiv AI

Jul 23

ENTRAP-VL: A Taxonomic Probe for Dual Contextual Entrainment in Vision-Language Models

arXiv:2607. 20092v1 Announce Type: cross Abstract: Contextual entrainment is the tendency of a model to let auxiliary context in its input pull its output, independently of whether that context is relevant, true, or even meaningful.

By Karan Goyal, Afreen Hossain, Debojyoti Das, Vishal Bhutani

llms multimodal benchmarks

arXiv Machine Learning

Jul 23

Hypothesis-and-Refinement Learning of Organic Structures from Multimodal Spectroscopic Data

arXiv:2607. 19816v1 Announce Type: cross Abstract: Determining molecular structures from spectroscopic data remains fundamentally challenging because the inverse problem is intrinsically underdetermined: individual spectra are sparse, low-dimensional, and encode only partial structural evidence relative to the vast space of possible molecules.

By Chengchun Liu, Zhiyuan Yan, Li Yuan, Hao Li, Boxuan Zhao, Yonghong Tian, Bartosz A. Grzybowski, Fanyang Mo

fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 23

Multi-modal transformer for signal classification in nanopore blockade experiments

arXiv:2607. 20323v1 Announce Type: new Abstract: Nanopore devices have emerged as powerful tools for single-molecule sensing, with potential for rapid, portable diagnostics.

By Sandro Kuppel, Julian Ho{\ss}bach, Samuel Tovey, Christian Holm

llms multimodal benchmarks

arXiv AI

Jul 23

Memory-Augmented Multimodal Large Language Models for Small Object Understanding in Streaming Aerial Videos

arXiv:2607. 19857v1 Announce Type: cross Abstract: Language-guided aerial perception aims to understand user-specified tiny targets in complex unmanned aerial vehicle (UAV) scenes.

By Penglei Sun, Yehua Huang, Zhuoli Tao, Xiang Li, Runwei Guan, Yaoxian Song, Kaiyong Zhao, Henghui Ding, Bo Han, Yang Yang, Xiaowen Chu

llms computer-vision multimodal

arXiv AI

Jul 23

Symbol and Footprint Database for Electronic Components by Agentic Recognition and Generation

arXiv:2607. 19767v1 Announce Type: new Abstract: A rich and recognizable component library is the cornerstone of printed circuit board (PCB) design and generation.

By Yichen Shi, Yuzhi Liu, Zhuofu Tao, Li Huang, Yuhao Gao, Ting-Jung Lin, Lei Hel

llms agents multimodal

Get the daily briefing