PACE: Pareto-Adaptive Compression for Efficient Native MLLMs

The paper introduces PACE, a Pareto‑adaptive compression framework designed for native multimodal large language models (MLLMs) that directly tokenize pixel patches without external vision encoders. By analyzing token discriminability and heterogeneous redundancy across spatial dimensions, PACE decomposes attention into height‑ and width‑specific saliency maps and applies a Pareto‑optimization strategy to allocate token budgets, achieving near‑lossless performance with only 25‑50% of visual tokens and more than 2× FLOP savings. Experiments demonstrate that PACE not only preserves but sometimes improves benchmark results compared to existing ViT‑based compression methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• multimodal
• benchmarks