arXiv AI
PW-Pruning: Phase-Aware Expert Selection for Speech-Centric Omni-Modal Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Oct 27
Efficient Omni-modal Large Language Model Inference via Per-Query Modality Selection
arXiv:2610.31768v1 Announce Type: new Abstract: Omni-modal large language models (Omni-LLMs) can jointly process a broad range of modalities, such as text, audio, and video, enabling stronger multimo...
arXiv Computation and Language
2d ago
Modality-Aware Expert Pruning for MoE-Based Multimodal Large Language Models
The paper introduces Modality-Aware Expert Pruning (MAEP), a training‑free method that decomposes expert importance by modality in a single forward pass for Mixture‑of‑Experts multimodal large language models. MAEP uses a Delta‑H Output metric to measure the impact of removing an expert and leverages low‑attention vision tokens as a stable pruning signal, while preserving text‑side importance. Experiments on three MoE‑based MLLMs show that MAEP reduces the BF16 expert weight footprint by about 27 GB (≈47% of total memory), enabling a 58 GB model to fit on a single 40 GB GPU and outperforming modality‑agnostic baselines on most multimodal benchmarks.
Hugging Face Trending Papers
Sep 22
From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs
As large language models (LLMs) scale rapidly, dense full-parameter adaptation becomes increasingly expensive, motivating sparse and modular architectures such as Mixture-of-Experts (MoE) models. This...
arXiv Computer Vision
Oct 15
OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Models
arXiv:2605.18041v2 Announce Type: replace Abstract: Omnimodal large language models (OmniLLMs) have recently gained increasing attention for unified audio-video understanding. However, processing lon...
arXiv Machine Learning
Jun 17
AIMER: Calibration-Free Task-Agnostic MoE Expert Pruning
arXiv:2603. 18492v3 Announce Type: replace Abstract: Mixture-of-Experts (MoE) language models increase parameter capacity without proportional per-token computation, yet deployment still requires storing the full expert pool, making expert pruning important for reducing memory and serving overhead.
arXiv AI
Jul 7
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
arXiv:2607. 03050v1 Announce Type: cross Abstract: Omni modal large language models (OmniLLMs) have attracted wide attention for their ability to jointly process audio and video, but they generate large token sequences under audio-visual inputs, leading to substantial inference cost.