arXiv Machine Learning June 2, 2026 By Zhiqiang Zhou, Xuezhen Xie

Feature Alignment Determines Fusion Strategy: A Comparative Study of Cross-Attention and Concatenation in Multimodal Learning

Read the original on arXiv Machine Learning →

arXiv:2606. 01207v1 Announce Type: cross Abstract: The choice between cross-attention and concatenation for multimodal fusion remains governed by practitioner intuition rather than principled understanding.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 14

PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging

arXiv:2604. 22823v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) rely on multimodal pre-training over diverse data sources, where different datasets often induce complementary cross-modal alignment capabilities.

By Zibo Shao, Baochen Xiong, Xiaoshan Yang, Yaguang Song, Qimeng Zhang, Haifeng Chen, Changsheng Xu

llms fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Aug 6

Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes

arXiv:2608. 05000v1 Announce Type: cross Abstract: Vision offers a critical axis for advancing foundation models, driving a shift towards natively unified multimodal pretraining.

By Junlin Han, Shengbang Tong, David Fan, Minghao Chen, Philip Torr, Filippos Kokkinos, Mike Lewis

nlp multimodal safety

arXiv Machine Learning

Aug 4

Hierarchical Pre-Training of Vision Encoders with Large Language Model

arXiv:2604. 00086v2 Announce Type: replace-cross Abstract: The field of computer vision has experienced significant advancements through scalable vision encoders and multimodal pre-training frameworks.

By Eugene Lee, Ting-Yu Chang, Jui-Huang Tsai, Jiajie Diao, Chen-Yi Lee

llms rag computer-vision multimodal benchmarks safety

arXiv AI

Jun 8

Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models

arXiv:2602. 07026v3 Announce Type: replace-cross Abstract: Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of distinct modalities expressing identical semantics occupy systematically offset regions.

By Xiaomin Yu, Yi Xin, Yuhui Zhang, Wenjie Zhang, Chonghan Liu, Hanzhen Zhao, Chen Liu, Xiaoxing Hu, Ziyue Qiao, Hao Tang, Xiaobin Hu, Chengwei Qin, Hui Xiong, Yu Qiao, Shuicheng Yan

llms rag fine-tuning multimodal safety

arXiv AI

Jul 16

Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping

arXiv:2508. 12466v2 Announce Type: replace-cross Abstract: Traditional multimodal learning approaches rely on alignment pre-training to bridge vision and language modalities, typically by projecting visual features into discrete text token spaces using large-scale image--text data.

By Xuhui Zhan, Tyler Derr

llms rag multimodal benchmarks safety

arXiv Machine Learning

Jun 3

Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals

arXiv:2606. 02679v1 Announce Type: new Abstract: Multimodal systems often benefit from combining information across language, sound, and visual streams, but this benefit is not guaranteed.

By Jiyuan Liu, Liangwei Nathan Zheng, Wei Emma Zhang, Xinpei Wang, Weitong Chen

multimodal benchmarks

Related stories

PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging

Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes

Hierarchical Pre-Training of Vision Encoders with Large Language Model

Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models

Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping

Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals