arXiv AI

MVCL-DAF++: Enhancing Multimodal Intent Recognition via Prototype-Aware Contrastive Alignment and Coarse-to-Fine Dynamic Attention Fusion

arXiv:2509. 17446v3 Announce Type: replace-cross Abstract: Multimodal intent recognition (MMIR) suffers from weak semantic grounding and poor robustness under noisy or rare-class conditions.

arXiv Machine Learning

Jun 5

GRAMformer: Any-Order Modality Interactions via Volumetric Multimodal Cross-Attention

arXiv:2606. 06249v1 Announce Type: cross Abstract: Transformer-based multimodal models rely on attention mechanisms to integrate information across heterogeneous modalities.

By Giordano Cicchetti, Eleonora Grassucci, Danilo Comminiello

llms multimodal

arXiv AI

Jun 3

CL-DMDF:Dynamic Multimodal Data Fusion Model Based on Contrastive Learning

arXiv:2606. 02659v1 Announce Type: cross Abstract: Multimodal data fusion involves integrating and analyzing information from multiple modalities to uncover latent correlations and complementary patterns, thereby enhancing data processing and decision-making.

By Dong Li, Lingling Zhang, Binghao Han, Linlin Ding, Yue Kou

multimodal

arXiv Machine Learning

1d ago

FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokens

arXiv:2506. 03096v2 Announce Type: replace-cross Abstract: Contrastive language-image pre-training aligns features of text-image pairs in a common latent space via distinct encoders for each modality.

By Christian Schlarmann, Francesco Croce, Nicolas Flammarion, Matthias Hein

llms rag diffusion multimodal

arXiv AI

Jun 2

Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition

arXiv:2606. 00959v1 Announce Type: new Abstract: Understanding modality interaction in multimodal large language models (MLLMs) is central to reliable deployment.

By Wanlong Fang, Tianle Zhang, Wen Tao, Alvin Chan

llms multimodal benchmarks safety

arXiv Machine Learning

Jul 2

Rosetta: Composable Native Multimodal Pretraining

arXiv:2607. 00293v1 Announce Type: cross Abstract: Achieving true artificial general intelligence requires foundation models capable of integrating new modalities without forgetting prior knowledge.

By Xiangyue Liu, Zijian Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Ping Tan

llms diffusion multimodal

arXiv AI

Aug 5

Adaptive Modality Reliability Diagnosis and Restoration for Robust Multimodal Intent Recognition

arXiv:2608. 03475v1 Announce Type: cross Abstract: Multimodal intent recognition combines linguistic, acoustic, and visual evidence, but individual modalities may be noisy, missing, semantically conflicting, or disproportionately dominant.

By Suraj Kumar, Mohnish Raj, Soumi Chattopadhayay, Chandranath Adak, Ayan Dutta

multimodal benchmarks

arXiv Machine Learning

Jun 2

Feature Alignment Determines Fusion Strategy: A Comparative Study of Cross-Attention and Concatenation in Multimodal Learning

arXiv:2606. 01207v1 Announce Type: cross Abstract: The choice between cross-attention and concatenation for multimodal fusion remains governed by practitioner intuition rather than principled understanding.

By Zhiqiang Zhou, Xuezhen Xie

llms multimodal safety

arXiv Machine Learning

Jun 3

Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals

arXiv:2606. 02679v1 Announce Type: new Abstract: Multimodal systems often benefit from combining information across language, sound, and visual streams, but this benefit is not guaranteed.

By Jiyuan Liu, Liangwei Nathan Zheng, Wei Emma Zhang, Xinpei Wang, Weitong Chen

multimodal benchmarks

arXiv Machine Learning

Jun 3

Reconstructing Content with Collaborative Attention for Universal Multimodal Representation Learning

arXiv:2603. 01471v3 Announce Type: replace-cross Abstract: Multimodal embedding models, rooted in multimodal large language models (MLLMs), have yielded significant performance improvements across diverse tasks such as retrieval and classification.

By Jiahan Chen, Da Li, Hengran Zhang, Yinqiong Cai, Lixin Su, Jiafeng Guo, Daiting Shi, Dawei Yin, Keping Bi

llms rag multimodal

arXiv AI

Jul 21

Learning to Detect Cross-Modal Negation: An Analysis of Latent Representations and an Attention-Based Solution

arXiv:2607. 17712v1 Announce Type: new Abstract: Detecting high-level semantic concepts like negation across modalities remains a challenge for current multimodal systems.

By Ali AbuSaleh, Leon Hammerla, Alexander Mehler

llms rag multimodal

arXiv AI

Jul 3

SEPS: Semantic-enhanced Patch Slimming Framework for fine-grained cross-modal alignment

arXiv:2511. 01390v2 Announce Type: replace-cross Abstract: Fine-grained cross-modal alignment aims to establish precise local correspondences between vision and language, forming a cornerstone for visual question answering and related multimodal applications.

By Xinyu Mao, Junsi Li, Haoji Zhang, Yu Liang, Ming Sun

llms diffusion nlp multimodal safety

arXiv AI

Jul 14

PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging

arXiv:2604. 22823v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) rely on multimodal pre-training over diverse data sources, where different datasets often induce complementary cross-modal alignment capabilities.

By Zibo Shao, Baochen Xiong, Xiaoshan Yang, Yaguang Song, Qimeng Zhang, Haifeng Chen, Changsheng Xu

llms fine-tuning multimodal benchmarks safety

Related stories