arXiv Machine Learning
SliMOO: Interpretable Multi-Objective Evolutionary Search for LLM Depth Pruning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 28
Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
arXiv:2607. 22583v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved widespread adoption because of their strong reasoning and query-response capabilities.
arXiv Computation and Language
Aug 31
Pruning Laws for Large Language Models
arXiv:2504.04342v2 Announce Type: replace Abstract: Scaling up model parameters and training data consistently improves the performance of large language models (LLMs), but at the cost of rapidly gro...
arXiv Machine Learning
Aug 4
GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Models
arXiv:2603. 13418v2 Announce Type: replace Abstract: Structured pruning is widely applied to compress large language models (LLMs), but its performance depends heavily on how neuron importance is estimated.
arXiv Computation and Language
2d ago
Modality-Aware Expert Pruning for MoE-Based Multimodal Large Language Models
The paper introduces Modality-Aware Expert Pruning (MAEP), a training‑free method that decomposes expert importance by modality in a single forward pass for Mixture‑of‑Experts multimodal large language models. MAEP uses a Delta‑H Output metric to measure the impact of removing an expert and leverages low‑attention vision tokens as a stable pruning signal, while preserving text‑side importance. Experiments on three MoE‑based MLLMs show that MAEP reduces the BF16 expert weight footprint by about 27 GB (≈47% of total memory), enabling a 58 GB model to fit on a single 40 GB GPU and outperforming modality‑agnostic baselines on most multimodal benchmarks.
arXiv Machine Learning
Jul 27
Layer-wise LoRA fine-tuning: a similarity metric approach
arXiv:2602. 05988v2 Announce Type: replace Abstract: Pre-training Large Language Models (LLMs) on web-scale datasets becomes fundamental for advancing general-purpose AI.
arXiv AI
5d ago
Prune Gradually, Adapt Sparsely: Efficient Pruning of Large Language Models at Extreme Sparsity
arXiv:2610.41968v1 Announce Type: new Abstract: Weight pruning can reduce the storage and inference cost of large language models (LLMs), but preserving quality at high sparsity levels remains challe...