arXiv AI
Mitigating Cross-Modal Hallucinations by Correcting Modality Misrouting in Omnimodal LLMs
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 5
Relevant Evidence Decoding for Audio-Visual Hallucination Mitigation
arXiv:2610.02976v1 Announce Type: new Abstract: Audio-Visual Large Language Models (AV-LLMs) remain prone to cross-modal hallucinations, where one modality incorrectly affects predictions about anoth...
arXiv AI
Sep 10
Tri-PvP: Exposing Modality Bias in Omni-Modal Large Language Models through Perceptual-Propositional Evidence Conflicts
arXiv:2609.06011v1 Announce Type: cross Abstract: Omni-modal large language models (OLLMs) jointly process vision, audio, and text, yet their modality bias under cross-modal conflict remains underexp...
Hugging Face Trending Papers
Sep 29
Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models
Audio-visual large language models (AVLLMs) have made remarkable progress in multimodal understanding and reasoning through interactions among visual, auditory, and linguistic information. However, re...
arXiv AI
Oct 21
Omni-Woodpecker: Hallucination Correction via Evidence-Prior Rebalancing for Omni LLMs
Omni-Woodpecker (OWP) is a training‑free framework designed to reduce cross‑modal hallucinations in Omni LLMs by rebalancing fused representations. It identifies conflicts between modality‑specific views, separates grounding evidence from competing priors via orthogonal decomposition, and adaptively suppresses misleading priors while restoring ignored evidence before decoding. Experiments show OWP improves performance on CMM, AVHBench, and several general reasoning benchmarks, and generalizes across different Omni LLM architectures.
arXiv Computation and Language
Oct 5
OmniConfess: Eliciting Token Confessions to Mitigate Omni-Modal Hallucination
OmniConfess is a training‑free method designed to reduce hallucinations in omni‑modal large language models (OmniLLMs) that handle text, images, audio, and video. The approach fixes a candidate response and re‑scores it at token resolution while selectively intervening on evidence from each modality, producing a token‑by‑channel confession that shows which evidence supports each part of the response. Using this confession, OmniConfess preserves grounded content and corrects commitments that rely on irrelevant or contradictory evidence. The authors evaluated the method on OmniHalluBench, a 3,540‑example benchmark drawn from six datasets across multiple modalities and tasks, and found that OmniConfess mitigates hallucinations across diverse settings.
arXiv AI
Oct 2
Senses Wide Shut: A Representation-Action Gap in Omnimodal LLMs
The paper introduces IMAVB, a 500‑clip benchmark that tests whether omnimodal large language models can detect when a textual premise contradicts their visual or audio input. Experiments on eight open‑source models and Gemini 3.1 Pro reveal a Representation‑Action Gap: internal states encode mismatches, yet the models rarely reject false premises, exhibiting under‑rejection or over‑rejection. A probe‑guided logit adjustment improves rejection behavior, suggesting the main bottleneck is in translating perception to action rather than in perception itself.