arXiv Computation and Language September 11, 2026 By Kaiyan Zhao, Zhongtao Miao, Akiko Aizawa, Yoshimasa Tsuruoka

FlexComp: One Model for Every Ratio in Context Compression

Read the original on arXiv Computation and Language →

FlexComp is a framework that allows a single model to perform context compression at any desired ratio, unlike existing methods that require separate models for each fixed ratio. It achieves this by sampling a memory budget during training and selecting the appropriate budget at inference time using either confidence-based cascade routing or a lightweight learned predictor. Experiments on ICAE, 500xCompressor, and SAC show that FlexComp matches the performance of specialized fixed-ratio models while enabling high compression rates and improving decoding throughput.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

Jun 29

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

arXiv:2606. 27866v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models scale model ability with sparsely activated experts, making this architecture a standard recipe for modern large models.

By Fan Mo, Yuxuan Han, Geng Zhang, Wangbo Zhao, Yang You

llms fine-tuning efficiency

arXiv AI

1d ago

Boosting Large Language Models with Mask Fine-Tuning

arXiv:2503.22764v3 Announce Type: replace-cross Abstract: The large language model (LLM) is typically integrated into the mainstream optimization protocol. However, it remains underexplored whether m...

By Mingyuan Zhang, Yue Bai, Huan Wang, Yizhou Wang, Qihua Dong, Yitian Zhang, Yun Fu

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 9

End-to-End Context Compression at Scale

arXiv:2606. 09659v1 Announce Type: cross Abstract: Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length.

By Ang Li, Sean McLeish, Haozhe Chen, Nimit Kalra, Zaiqian Chen, Artem Gazizov, Venkata Anoop Suhas Kumar Morisetty, Bhavya Kailkhura, Harshitha Menon, Zhuang Liu, Brian R. Bartoldson, Tom Goldstein, Sanae Lotfi, Micah Goldblum, Pavel Izmailov

llms rag agents efficiency

arXiv Machine Learning

Sep 17

Beyond Static RAG: An Adaptive, Tri-Metric Routing Framework for Efficient Long-Context Inference on Commodity GPUs

The paper introduces the Tri‑Metric Router, a deterministic, training‑free policy that chooses among Raw, Neural, and Lexical pipelines for retrieval‑augmented generation on commodity GPUs. It uses three CPU‑side signals—spatial complexity, syntactic density, and type‑token ratio—to balance VRAM headroom and latency, calibrated on LongBench qasper. The method eliminates out‑of‑memory failures and improves alignment and F1 scores compared to always‑on lexical compression without extra VRAM or training costs.

By Saipraveen Vabbilisetty, Ajay Kumar Boddepalli, Deep Narayan Mishra, Shashank Kapadia, Haoan Wang, Anupriya Sharma

llms rag safety

Hugging Face Trending Papers

Aug 25

Compression Trinity: Exploring Sparsity, Quantization, and Low-Rank Approximations for LLM Compression

Prohibitive computational and environmental costs impede the scalable deployment of Large Language Models (LLMs). Traditional compression techniques (sparsity, quantization, low-rank approximations) a...

llms fine-tuning efficiency benchmarks

Hugging Face Trending Papers

Jun 8

End-to-End Context Compression at Scale

Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require considerable time and compute to compress a single long prompt.

llms rag agents efficiency

One email a morning, machine-written

Related stories

FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

Boosting Large Language Models with Mask Fine-Tuning

End-to-End Context Compression at Scale

Beyond Static RAG: An Adaptive, Tri-Metric Routing Framework for Efficient Long-Context Inference on Commodity GPUs

Compression Trinity: Exploring Sparsity, Quantization, and Low-Rank Approximations for LLM Compression

End-to-End Context Compression at Scale