arXiv AI
Per-Matrix Optimality Is Not Enough: Three-Level Optimization for Low-Rank LLM Compression
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
1d ago
Learning Functional Subspaces for Neural Network Compression
Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keeping the matrices dense, and thus efficient on standar...
arXiv Computation and Language
1d ago
Learning Functional Subspaces for Neural Network Compression
arXiv:2609.40127v1 Announce Type: cross Abstract: Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keepin...
arXiv AI
Sep 11
Scaling Post-Training Ternarisation to Qwen3-8B Capability Retention, Reproduction, Lossless Packing, and Packed Execution
The paper reports a large‑scale post‑training ternarisation of the Qwen3 language model, extending a conversion pipeline from the 4B to the 8B variant. Using KOTMS rotation, E2M‑ATQ adaptive ternarisation, and GPTQ‑style error compensation, the authors achieve a 1.361× perplexity ratio across three corpora and retain 78.5% of the FP16 accuracy on zero‑shot tasks, with the 8B model outperforming the 4B by 8.9 percentage points. The study also demonstrates lossless lattice‑aware packing, producing an 8.24 GiB checkpoint that preserves perplexity, and shows that direct packed execution can reach 15.52 tokens/s in 7.35 GiB, though packed GEMV remains slower than FP16 cuBLAS.
arXiv AI
Jun 2
From Layers to Submodules: Rethinking Granularity in Replacement-Based LLM Compression
arXiv:2606. 02559v1 Announce Type: cross Abstract: Post-training compression of Large Language Models (LLMs) removes entire architectural components, either deleting them or replacing them with fitted modules.
arXiv AI
Sep 2
QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization
arXiv:2609.00224v1 Announce Type: cross Abstract: Weight-only post-training quantization (PTQ) can alleviate the computational burden of serving large language models (LLMs) at scale. However, existi...
arXiv Machine Learning
Sep 3
XMerge: Cross-Axis Selection and Reconstructive Layer Merging for LLM Depth Compression
XMerge is a post‑training method for compressing large language models by removing entire transformer layers while preserving a standard serving architecture. It selects low‑impact blocks via cross‑axis selection and refits adjacent surviving blocks with local boundary reconstruction, requiring no task labels or fine‑tuning. Across seven Llama and Qwen backbones, XMerge outperforms five published baselines, achieving top rankings on CORE and MMLU tasks even at aggressive compression levels, and consistently avoids model collapse while improving calibration and decoding efficiency.