Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

8,964 stories · RSS feed

arXiv AI

Jul 15

Lost in Visual Translation: A VLM-Assisted Perceptual-Semantic Coherence Framework for EEG-to-Image Reconstruction

arXiv:2607. 12364v1 Announce Type: cross Abstract: EEG-to-image evaluation should distinguish visual fidelity from recoverable meaning.

By Sukriti Tiwari, BHVSP Subrahmanyam, Nidhi Goyal, Sai Amrit Patnaik

multimodal safety

arXiv Machine Learning

Jul 15

Ensemble Controlled-Flow Filtering for Implicit Data Assimilation

arXiv:2607. 12975v1 Announce Type: cross Abstract: Data assimilation estimates the state of a dynamical system from model forecasts and incoming observations.

By Zhuoyuan Li, Yue Zhao, Ming Li

multimodal

arXiv Machine Learning

Jul 15

Hierarchical Synthetic Tabular Data Generation: A Hybrid Top-Down and Bottom-Up Framework

arXiv:2605. 28198v2 Announce Type: replace Abstract: Existing approaches for synthetic tabular data generation are based on either purely generative models or LLMs, both of which struggle with data heterogeneity, logical consistency, rare-event coverage, and robustness in low-data regimes.

By Junfeng Nie, Alvin Jin, Xiaohui Chen

llms multimodal benchmarks safety

arXiv Machine Learning

Jul 15

Spectral Diffusion Processes

arXiv:2209. 14125v3 Announce Type: replace-cross Abstract: Diffusion models have proven to be a flexible and effective framework for modelling probability distributions on finite-dimensional spaces.

By Angus Phillips, Thomas Seror, Michael Hutchinson, Valentin De Bortoli, Arnaud Doucet, Emile Mathieu

diffusion multimodal

arXiv AI

Jul 15

PFAdapter: Hierarchical LoRA Decomposition for Personalized Federated MLLMs

arXiv:2607. 12111v1 Announce Type: cross Abstract: Agentic AI systems are reshaping communications and networking by deploying autonomous intelligent agents capable of collaborative learning while maintaining data privacy at network edges.

By Jing Liu, Kun Yang, Yan Wang, Dingkang Yang, Xiaoshuai Hao, Wei Zhang, Yang Liu, Wei Zhou

llms agents fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jul 15

Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarks

arXiv:2607. 12278v1 Announce Type: cross Abstract: Recent vision-language models (VLMs) for computational pathology report striking zero-shot performance on whole-slide image (WSI) visual question answering (VQA) benchmarks.

By Wenhao Zhang, Zhongliang Zhou, John Kang, Sheng Li

llms nlp multimodal benchmarks

arXiv AI

Jul 15

Visual Access Boundaries in Vision-Language Model Reasoning

arXiv:2607. 12815v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting is widely used as a test-time scaling strategy for Vision-Language Models (VLMs), but it remains unclear what is extended when VLMs generate longer reasoning traces.

By Hiroto Osaka, Shohei Taniguchi, Gouki Minegishi, Kai Yamashita, Masahiro Suzuki, Yutaka Matsuo

llms multimodal

arXiv AI

Jul 15

Text-Aided Multi-Modal Panoptic Symbol Spotting for CAD Floor Plan Drawings

arXiv:2607. 12678v1 Announce Type: cross Abstract: Computer-Aided Design (CAD) floor plan drawings contain both graphical primitives and textual annotations, which provide complementary geometric and semantic cues for intelligent design understanding.

By Yan Gong, Bohao Li, Bowen Du, Junchen Ye

multimodal benchmarks safety

arXiv AI

Jul 15

IQA-T1: Tool-based Visual Evidence Reasoning for Image Quality Assessment

arXiv:2607. 12375v1 Announce Type: cross Abstract: Image Quality Assessment (IQA) in open-world environments remains challenging due to limited generalization and interpretability.

By Jinjian Wu, Jiaqi Tang, Wei Wei, Yingying Yan, Jianmin Chen, Botong Geng, Lei Zhang, Qifeng Chen

llms multimodal benchmarks safety

arXiv AI

Jul 15

HSEmotion Team at the 11th ABAW Challenge: Multi-Task Learning and Ambivalence/Hesitancy Video Recognition

arXiv:2607. 12774v1 Announce Type: cross Abstract: This article presents our results for the 11th Affective Behavior Analysis in-the-Wild (ABAW) competition.

By Aleksei Bakin, Andrey V. Savchenko

fine-tuning multimodal safety

arXiv AI

Jul 15

Xray-Visual Models: Scaling Vision models on Industry Scale Data

arXiv:2602. 16918v2 Announce Type: replace-cross Abstract: We present Xray-Visual, a unified vision model architecture for large-scale image and video understanding trained on industry-scale social media data.

By Shlok Mishra, Tsung-Yu Lin, Linda Wang, Hongli Xu, Yimin Liu, Michael Hsu, Chaitanya Ahuja, Hao Yuan, Jianpeng Cheng, Hong-You Chen, Haoyuan Xu, Chao Li, Sreya Dutta Roy, Abhijeet Awasthi, Jihye Moon, Don Husa, Michael Ge, Sumedha Singla, Arkabandhu Chowdhury, Phong Dingh, Satya Narayan Shukla, Yonghuan Yang, David Jacobs, Qi Guo, Jun Xiao, Xiangjun Fan, Aashu Singh

llms computer-vision multimodal benchmarks safety

arXiv AI

Jul 15

A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its Mechanism

arXiv:2607. 12640v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards, and Group Relative Policy Optimization (GRPO) in particular, is now run routinely on a supervised checkpoint in the hope of producing a stronger agent.

By Chengguang Gan, Zhixi Cai, Yunhao Liang, Hanjun Wei, Shiwen Ni, Qinghao Zhang

llms rag agents reinforcement-learning multimodal

arXiv AI

Jul 15

DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter

arXiv:2602. 05513v3 Announce Type: replace-cross Abstract: Bimanual dexterous manipulation relies on integrating multimodal inputs to perform complex real-world tasks.

By Xukun Li, Yu Sun, Lei Zhang, Bosheng Huang, Yibo Peng, Yuan Meng, Haojun Jiang, Shaoxuan Xie, Guocai Yao, Alois Knoll, Zhenshan Bing, Xinlong Wang, Zhenguo Sun

llms diffusion robotics multimodal

arXiv AI

Jul 15

Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering

arXiv:2603. 28583v2 Announce Type: replace-cross Abstract: Despite the success of Vision-Language Models (VLMs), misleading charts remain a significant challenge due to their deceptive visual structures and distorted data representations.

By Yanjie Zhang, Yafei Li, Rui Sheng, Zixin Chen, Yanna Lin, Huamin Qu, Lei Chen, Yushi Sun

llms agents computer-vision nlp fine-tuning efficiency multimodal benchmarks safety

arXiv Machine Learning

Jul 15

ANGLE: Angular Neural Generative Learning via Engression

arXiv:2607. 12833v1 Announce Type: cross Abstract: Circular data, representing angles or directions, are frequently encountered in computer vision, biology, geology, and meteorology.

By Rajdeep Pathak, Archi Roy, Tanujit Chakraborty

agents computer-vision multimodal

arXiv AI

Jul 15

Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference

arXiv:2607. 12659v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have achieved impressive performance on diverse embodied tasks.

By Zebin Yang, Qi Wang, Yunhe Wang, Xiurui Guo, Bo Yu, Shaoshan Liu, Jiafeng Xu, Hao Dong, Meng Li

llms robotics efficiency multimodal benchmarks

arXiv AI

Jul 15

Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

arXiv:2509. 22415v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved strong vision-language performance, yet their token-level visual evidence remains difficult to inspect.

By Jiawei Liang, Jianjie Huang, Ruoyu Chen, Xianghao Jiao, Siyuan Liang, Shiming Liu, Xiaochun Cao

llms multimodal

arXiv AI

Jul 15

NeSy-Route: A Neuro-Symbolic Benchmark for Constrained Route Planning in Remote Sensing

arXiv:2603. 16307v2 Announce Type: replace Abstract: Remote sensing underpins crucial applications such as disaster relief and ecological field surveys, where systems must understand complex scenes and constraints and make reliable decisions.

By Ming Yang, Zhi Zhou, Shi-Yu Tian, Kun-Yang Yu, Lan-Zhe Guo, Yu-Feng Li

llms multimodal benchmarks

arXiv AI

Jul 15

The Emerging Paradigm of Geospatial Foundation Models: From Pre-Training to Agentic Reasoning

arXiv:2607. 12177v1 Announce Type: new Abstract: The analysis of satellite and aerial imagery has entered a new era with the advent of foundation models.

By Shelley Cazares

llms agents fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 15

Beyond Logit Adjustment: A Residual Decomposition Framework for Long-Tailed Reranking

arXiv:2604. 01506v2 Announce Type: replace Abstract: Long-tailed classification, where a small number of frequent classes dominate many rare ones, remains challenging because models systematically favor frequent classes at inference time.

By Zhanliang Wang, Hongzhuo Chen, Quan Minh Nguyen, Mian Umair Ahsan, Kai Wang

rag nlp multimodal benchmarks

Get the daily briefing