arXiv:2609.15320v1 Announce Type: new
Abstract: Volume-based multimodal retrieval jointly scores a text query with a candidate's video, audio, and subtitle embeddings. While this approach captures hi...
By Anindya Nag, Ambuj Mehrish, Sebastiano Vascon
arXiv:2610.35125v1 Announce Type: new
Abstract: Multimodal video retrieval combines visual, acoustic, and speech information, yet how their fusion order should depend on the data and the retrieval qu...
By Haoxiang Zhao, Bin Wang, Yaofei Wang, Xiao Lan, Yunfeng Diao, Xu Wang
arXiv:2608.23484v1 Announce Type: new
Abstract: We present Team Semiintelligencn's solution for the ACM RecSys 2026 TalkPlayData Challenge, addressing conversational music recommendation through a mu...
By Naman Garg, Sarika Jain, George Fazekas
We present Team Semiintelligencn's solution for the ACM RecSys 2026 TalkPlayData Challenge, addressing conversational music recommendation through a multi-modal and personalized conversational recomme...
The paper describes the CERTH-ITI team’s approach to the TRECVID 2025 Ad-hoc Video Search (AVS) and Video Question Answering (VQA) tracks. For AVS, a two-stage framework uses foundation vision‑language models for initial retrieval and a Multimodal‑LLM for reranking, improving contextual matching without training. In VQA, an audio‑visual MLLM is fine‑tuned and enhanced with inference‑time scaling; answer generation employs majority voting over responses from random frame subsets, while multiple‑choice responses use mean‑pooling logits, yielding top performance on VQA metrics.
By Andreas Goulas, Damianos Galanopoulos, Ioannis Patras, Vasileios Mezaris
arXiv:2607. 25266v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have enabled long-form video understanding at a scale that was not previously possible.
By Ghazal Kaviani, Ghassan AlRegib
The paper investigates how retrieval heads—sparse attention heads that transfer information from context to output—behave when multimodal large language models (MLLMs) are extended to include audio-visual inputs. Across 18 language, vision‑language, and omni‑modal checkpoints, the authors find that retrieval heads remain sparse and largely shared across modalities, with less than 2 % of heads being common to all modalities; ablating these shared heads degrades tasks such as document understanding, spatial perception, and visual mathematics. By training retrieval heads with a ranking objective, they demonstrate a 6.03‑point improvement in Recall@1 on the MMDocIR benchmark for Qwen3‑VL‑8B and improved document retrieval across three model families.
By Jia Wang, Yuqing Li, Jiangnan Li, Zheng Lin, Mo Yu
arXiv:2608. 12987v1 Announce Type: cross Abstract: Generative information retrieval (GIR) has emerged as a compelling alternative to the conventional index-retrieve-then-rank retrieval pipeline by training a generator to produce the identifiers of relevant items directly.
By Kaipeng Li, Haitao Yu, Xuanchen Zhou
arXiv:2609.39836v1 Announce Type: new
Abstract: Contrastive vision-language models map visual and textual representations into a shared normalized embedding space, making cosine similarity the natura...
By Simone Ricci, Niccol\`o Biondi, Federico Pernici
arXiv:2609.10224v1 Announce Type: new
Abstract: Vision-language models such as CLIP embed images and text in a shared space, where modality-specific distributions often remain separated. Existing acc...
By Zonglin Yang, Huilan Ma, Xudan Zheng, Yuejun Xie
The paper introduces D3-Omni, a balanced and decoupled benchmark designed to diagnose fine‑grained multimodal understanding in OmniJudges that evaluate text‑to‑image, text‑to‑video, and text‑to‑speech generation. D3-Omni covers 53 orthogonal binary dimensions across 10,671 samples, using fixed positive seeds and controlled prompt rewriting to generate negatives, thereby ensuring each error can be attributed to a single capability. The benchmark’s dual‑balanced, decoupled, and dynamic design achieves near 1:1 per‑dimension parity and a uniform total‑score distribution, revealing that strong OmniJudges often miss modality‑related failures and treat distinct attributes as a single decision, masking systematic blind spots.
By Guangzheng Hu, Ziyue Jiang, Weixu Qiao, Lixin Zhang, Jianye Kang, Yuru Wu, Rong Bao, Niantong Li, Wei Wang, Ziyi Cheng, Xinfa Zhu, HangRui Hu, Ting He, Bing Zhao, Lin Qu, Hu Wei, Jin Xu
arXiv:2603.00720v2 Announce Type: replace
Abstract: Fine-tuning Multimodal Large Language Models (MLLMs) with parameter-efficient methods like Low-Rank Adaptation (LoRA) is crucial for task adaptatio...
By Minkyoung Cho, Insu Jang, Shuowei Jin, Zesen Zhao, Adityan Jothi, Ethem F. Can, Min-Hung Chen, Z. Morley Mao