arXiv Machine Learning
Efficient Omni-modal Large Language Model Inference via Per-Query Modality Selection
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 8
Modality Relevance is not Modality Utility: Post-hoc Selective Modality Escalation for Cost-Aware Multimodal RAG
arXiv:2607. 05438v1 Announce Type: cross Abstract: Multimodal retrieval-augmented generation (RAG) grounds a generator in evidence drawn from heterogeneous modalities -- text, tables, and images.
arXiv AI
Jul 7
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
arXiv:2607. 03050v1 Announce Type: cross Abstract: Omni modal large language models (OmniLLMs) have attracted wide attention for their ability to jointly process audio and video, but they generate large token sequences under audio-visual inputs, leading to substantial inference cost.
arXiv AI
Jul 29
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
arXiv:2607. 25669v1 Announce Type: new Abstract: Emerging Omni-modal Large Language Models (OmniLLMs) enable unified understanding of text, audio, and video, but their long audio-video token sequences introduce substantial memory and inference costs.
arXiv AI
Aug 11
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
arXiv:2608. 09227v1 Announce Type: new Abstract: Omnimodal language models (OLMs) enable unified audio-visual understanding, but processing long joint token sequences makes inference computationally prohibitive.
arXiv Computer Vision
Oct 15
OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Models
arXiv:2605.18041v2 Announce Type: replace Abstract: Omnimodal large language models (OmniLLMs) have recently gained increasing attention for unified audio-video understanding. However, processing lon...
arXiv Computation and Language
Oct 23
TRIAD: Benchmarking Omni-Modal Ambiguity in Multimodal Large Language Models
TRIAD is a new benchmark designed to test omni‑modal large language models’ ability to resolve ambiguity across text, vision, and audio. Each item consists of a text–image–audio triplet, a question, and answer options, constructed so that the full triplet yields a unique answer while any single modality removed makes the answer ambiguous. The benchmark includes 327 queries in 86 scene groups, covering 18 ambiguity categories, and shows that current models lag humans in scene‑level disambiguation, especially when answer options are removed.