arXiv Machine Learning
Fine-Tuning Low-Bit Models with Gradient in Quantized Code Space
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 8
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
arXiv:2605. 08692v2 Announce Type: replace Abstract: Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference.
arXiv AI
Aug 6
Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs
arXiv:2608. 04048v1 Announce Type: cross Abstract: Serving large language models (LLMs) under diverse deployment constraints requires flexible trade-offs between accuracy, memory footprint, and throughput.
arXiv AI
Jun 10
LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization
arXiv:2606. 10531v1 Announce Type: cross Abstract: Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs).
arXiv AI
Jun 15
Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost
arXiv:2602. 03120v2 Announce Type: replace-cross Abstract: Post-Training Quantization (PTQ) is essential for deploying Large Language Models (LLMs) on memory-constrained devices, yet it renders models static and difficult to fine-tune.
arXiv Machine Learning
Jul 3
QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs
arXiv:2602. 10431v4 Announce Type: replace Abstract: Large language models (LLMs) demand substantial computational and memory resources, posing challenges for efficient deployment.
arXiv AI
Jun 4
Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data
arXiv:2606. 04238v1 Announce Type: cross Abstract: Aggressive weight quantization to 2-bit precision offers substantial throughput and memory gains for large language model (LLM) inference, but typically incurs severe accuracy degradation.