The paper introduces OnQ, a low‑bit recovery pipeline for quantized large language models that uses on‑policy distillation and candidate‑wise Dynamic KL to address trajectory divergence and distributional misalignment. OnQ starts with block‑wise calibration, then lets the quantized student generate responses and receive full‑precision teacher supervision on its own prefixes, dynamically weighting forward and reverse KL losses per candidate. Experiments on Qwen3 models show that OnQ outperforms baselines on nine benchmarks, especially long‑horizon reasoning, and produces shorter, less truncated responses.
By Boyuan Zheng, Jing Xiong, Chenwei Wu, Hui Shen, Zesen Zhao, Hei Ting Una Chan, Xueshen Liu, Zhen Zhang, Liyue Shen, Zechun Liu, Z. Morley Mao
The paper introduces Weakening-Induced On-Policy Self-Distillation (WISD), a method that improves large reasoning models by creating a weaker version of the original model and using the contrast between them as a training signal, without needing privileged information. WISD constructs the weaker model via low-bit quantization or magnitude pruning, and evaluates it on mathematical and coding reasoning tasks across various architectures and scales. The results show significant accuracy gains—up to 5.5 percentage points—over the original models, achieving state‑of‑the‑art performance on benchmarks such as HMMT 2025 and LiveCodeBench v6.
By Yecheng Wu, Song Han, Han Cai
arXiv:2609.26708v1 Announce Type: new
Abstract: Quantization-aware distillation (QAD) restores much of the short-form question-answering performance lost to sub-3-bit quantization, yet leaves mathema...
By Yuanteng Chen, Zhilei Liu, Peisong Wang, Yuantian Shao, Chuangyi Li, Weining Wang, Shuang Qiu, Gang Li, Jing Liu, Jian Cheng
arXiv:2610.28537v1 Announce Type: new
Abstract: Post-training quantization (PTQ) compresses the weights of a trained model with limited compute and data. Down to 4 bits it is near-lossless, but at 2...
By Rhea Sanjay Sukthanker, Riccardo Grazzi, James Hensman, Pashmina Cameron
arXiv:2606. 15682v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) achieve strong problem-solving through long chain-of-thought, but their deployment is constrained by the high cost of full-precision inference and growing KV cache footprints.
By Janghwan Lee, Sihwa Lee, Jinseok Kim, Yongjik Kim, Jieun Lim, Jinwook Oh, Jungwook Choi
arXiv:2606. 02011v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive.
By Ekaterina Alimaskina, Darya Rudas, Denis Shveykin, Gleb Molodtsov, Pavel Vasiliev, Aleksandr Beznosikov