arXiv:2608. 05381v1 Announce Type: new Abstract: Current Multimodal Large Language Models (MLLMs) can process diverse sensory inputs, yet their reasoning remains heavily biased toward a dominant modality, resulting in brittle cross-modal reasoning.
arXiv:2609.36798v1 Announce Type: cross Abstract: Omni-modal large language models (LLMs) are expected to answer a question using the modality it explicitly refers to. However, existing training para...
arXiv:2610.18141v1 Announce Type: new Abstract: Omnimodal large language models (OLLMs) are susceptible to cross-modal hallucinations under audio-visual interference. We find that, for a substantial...
Video-HolmesV2 is a new benchmark that tests multimodal large language models on their ability to reason with spatio‑temporal audio‑visual evidence in long videos. It requires models to justify answers with precise evidence, uses a multi‑model cross‑verification pipeline and a spatio‑temporal evidence‑aware metric, and introduces an audio‑text guided token compression framework to reduce long‑context noise. In evaluations, even strong proprietary models score below 60% while the proposed approach outperforms comparable open‑source omni‑models.
The paper introduces IMAVB, a 500‑clip benchmark that tests whether omnimodal large language models can detect when a textual premise contradicts their visual or audio input. Experiments on eight open‑source models and Gemini 3.1 Pro reveal a Representation‑Action Gap: internal states encode mismatches, yet the models rarely reject false premises, exhibiting under‑rejection or over‑rejection. A probe‑guided logit adjustment improves rejection behavior, suggesting the main bottleneck is in translating perception to action rather than in perception itself.
The paper introduces Synergistic Faithfulness (𝔽_syn), a new metric based on the Shapley Interaction Index that isolates the joint contribution of visual and textual modalities in Vision‑Language Models (VLMs). It demonstrates that existing explainers overemphasize visual salience and fail to capture true cross‑modal synergy, and provides a scalable evaluation framework for auditing VLM reasoning. The authors evaluate nine XAI methods across three VLM architectures and datasets, revealing significant shortcomings in current approaches.
The paper investigates whether the layer that yields the highest probing accuracy in omni‑modal large language models is also the most effective for steering interventions. Across three independently developed models, the authors find that the best probing layers differ widely, whereas the most steerable layers consistently lie in a narrow mid‑to‑late range of the network. Using emotion as a testbed, they demonstrate a significant causal gap between probing and steering, and propose a two‑factor account linking readability and downstream plasticity to steering effectiveness.
Multimodal Large Language Models have demonstrated impressive video understanding, yet their ability to reason over long-form narratives is often masked by visual-centric evaluations and inefficient c...
OmniHallu is a unified framework for detecting hallucinations in multimodal large language models across both comprehension and generation tasks involving image, video, and audio modalities. It introduces OmniHallu-Bench, a 10,000-sample benchmark with claim-level human annotations for six cross-modal tasks (I2T, V2T, A2T, T2I, T2V, T2A). The system uses a multi‑agent architecture that decomposes outputs into atomic claims, verifies them with modality‑specific experts, and aggregates evidence through structured reasoning, while a preference‑optimized verifier reduces expert calls by 66% with minimal performance loss.
arXiv:2608. 04509v1 Announce Type: new Abstract: Vision-language systems combine images with retrieved text, but these sources can disagree or jointly fail to support an answer.
arXiv:2608. 06938v1 Announce Type: cross Abstract: The visual reasoning ability of multimodal large language models (MLLMs) is crucial for downstream applications, particularly counter-commonsense reasoning, which requires models to reason beyond common assumptions.
The paper introduces MuPHI, a diagnostic benchmark that isolates multimodal harm arising solely from image‑text composition, providing benign counterfactuals and annotated harm rationales. Evaluation with MuPHI shows that vision‑language models can ground individual modalities but struggle to bind evidence into a coherent harm mechanism, and that existing harm detection methods transfer poorly to new harm distributions. To address this, the authors propose Compositional Harm Interpretation Learning (CHIL), which rewards evidence identification, cross‑modal binding, and verdict consistency, leading to significant improvements in out‑of‑distribution transfer and counterfactual robustness.