The paper introduces SCALAR, a query‑conditioned spherical centroid aggregator that assigns relevance‑based weights to each available modality before computing a spherical centroid. SCALAR supports arbitrary modality subsets, is trained with rank‑8 LoRA adapters on masked views, and achieves positive aggregation gains on four of five benchmarks, outperforming prior symmetric aggregators. With only 4.8 million trainable parameters, SCALAR attains the highest text‑to‑video R@1 on three benchmarks and surpasses the released GRAM checkpoint under test‑time modality dropout by 3.2 to 10.9 R@1.
By Ambuj Mehrish, Anindya Nag, Sebastiano Vascon
The Modality Maturity Index (MMI) is a new benchmark that evaluates large language models on their ability to handle five different modalities—text, image, audio, video, and document—across up to three-input and three-output combinations. It contains 893 self‑contained questions, each with human‑authored rubric criteria for the expected output modalities, and measures performance via an MMI Value and a Modality Presence Score (MPS). Experiments on five frontier multimodal models show low MPS scores, indicating limited modality availability, and confirm that LLM judges can reliably assess output correctness against human‑blind rubric scoring on 70.8% of cases.
By Rohit Patel, Dieuwke Hupkes, Sloan Strader
arXiv:2609.37114v1 Announce Type: new
Abstract: DANCo (Dimensionality from Angle and Norm Concentration) jointly calibrates nearest-neighbor distance and angular statistics and consistently reaches s...
By Chih-Hsuan Huang, Chih-Wei Chen, Szu-Chi Chung
The paper investigates how kernel-based metrics like KID and KAD perform as two‑sample tests on real embeddings of symbolic music, audio, and images. By applying mean shifts, isotropic contractions, and rotations to one sample, the authors find that the default kernels often fail to detect rotations at 780 samples per side, while they reliably detect mean shifts. Adjusting kernel bandwidths or using alternative kernels (e.g., narrower Gaussian, Laplace) restores sensitivity, and the authors recommend reporting the sample budget and detectable changes for any reported kernel distance.
By Tomasz Radzikowski, Mateusz Modrzejewski
DANCo (Dimensionality from Angle and Norm Concentration) jointly calibrates nearest-neighbor distance and angular statistics and consistently reaches state-of-the-art accuracy on clean intrinsic-dimen...
arXiv:2606. 11190v1 Announce Type: new Abstract: Cross-modal alignment (CA) and cross-modal prediction (CP) are the dominant paradigms for multimodal representation learning, yet there is no systematic understanding of when each succeeds, when each fails, and when cross-modal training helps at all -- a gap that leaves practitioners, especially in scientific domains like biomedicine or astrophysics, with heterogeneous instruments and multiple levels of organization and measurement, unable to diagnose why standard methods underperform the best single modality.
By Ilay Kamai, Hugues Van Assel, Aviv Regev, Hagai B. Perets, Randall Balestriero
arXiv:2608.24881v1 Announce Type: cross
Abstract: Generative models are commonly ranked by Fr\'echet Inception Distance (FID) and Kernel Inception Distance (KID), yet FID's first-two-moment summary c...
By Hao Chen
arXiv:2609.15320v1 Announce Type: new
Abstract: Volume-based multimodal retrieval jointly scores a text query with a candidate's video, audio, and subtitle embeddings. While this approach captures hi...
By Anindya Nag, Ambuj Mehrish, Sebastiano Vascon
ANCHOR is a new benchmark for audio‑visually grounded chain‑of‑thought reasoning, containing 2,319 videos and 8,272 question‑answer pairs with structured reasoning traces, spatial bounding boxes, temporal intervals, and audio cues. It evaluates not only answer correctness but also hallucination, logical divergence, and conciseness, providing direct spatial and temporal grounding measurements on a focused subset. The benchmark reveals a substantial alignment gap among contemporary multimodal large language models, showing that similar answer accuracy can mask differences in evidence‑supported explanations.
By Joel Julin, Souraja Kundu, Liza Dahiya, George Z. Wei, Liuyue Xie, Jingru Yi, Xu Zhang, Hao Yang, Laszlo A. Jeni
arXiv:2607. 11007v1 Announce Type: new Abstract: Few-shot multimodal classification commonly attaches a lightweight head, such as $k$-nearest neighbors, logistic regression, or a linear SVM, to a frozen pretrained encoder.
By Jingxiang Zhang, Lujia Zhong, Zijie Zhu, Shuo Huang, Yuang Xu
arXiv:2605. 00873v2 Announce Type: replace-cross Abstract: The rapid advancement of photorealistic Text-to-Video (T2V) generation brings in an urgent need for up-to-date evaluation methods.
By Advait Tilak, Jiwon Choi, Nazifa Mouli, Wei Le
PRISM‑Bench is an audio‑centric diagnostic benchmark for text‑to‑audio‑video generation, built from 900 human‑verified samples. It evaluates audio along two axes—audio type (speech, music, sound) and sound‑source visibility (on‑screen vs. off‑screen)—across four perceptual dimensions (audio‑visual coherence, audio quality, audio expressiveness, and prompt following) using 35 fine‑grained criteria. The benchmark employs an enhanced MLLM‑as‑a‑Judge protocol that aligns strongly with human raters, revealing a performance gap between frontier and open‑source T2AV models and highlighting overfitting to perceptual fidelity while struggling with complex grounding and control tasks, especially for music and synchronized on‑screen audio.
By Yuchen Sun, Qian Yang, Jun Wang, Detai Xin, Guoqiao Yu, Guanglu Wan, Qi Jia