Hugging Face Trending Papers June 1, 2026

Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery

Read the original on Hugging Face Trending Papers →

Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive. While low-bit quantization reduces per-token decoding cost, we show that aggressive 2-bit inference can fail to deliver end-to-end speedup because instability in the generation process inflates total token count.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI

Jun 2

Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery

arXiv:2606. 02011v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive.

By Ekaterina Alimaskina, Darya Rudas, Denis Shveykin, Gleb Molodtsov, Pavel Vasiliev, Aleksandr Beznosikov

efficiency benchmarks

arXiv Machine Learning

Sep 23

Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning

arXiv:2609.26708v1 Announce Type: new Abstract: Quantization-aware distillation (QAD) restores much of the short-form question-answering performance lost to sub-3-bit quantization, yet leaves mathema...

By Yuanteng Chen, Zhilei Liu, Peisong Wang, Yuantian Shao, Chuangyi Li, Weining Wang, Shuang Qiu, Gang Li, Jing Liu, Jian Cheng

efficiency benchmarks safety

arXiv Machine Learning

Jun 16

ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training

arXiv:2606. 15682v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) achieve strong problem-solving through long chain-of-thought, but their deployment is constrained by the high cost of full-precision inference and growing KV cache footprints.

By Janghwan Lee, Sihwa Lee, Jinseok Kim, Yongjik Kim, Jieun Lim, Jinwook Oh, Jungwook Choi

fine-tuning efficiency data-engineering

arXiv Machine Learning

Jun 25

Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models

arXiv:2606. 25519v1 Announce Type: cross Abstract: Quantization is widely used to reduce the inference cost of large language models, but its effect on reasoning models is not fully captured by final-answer accuracy or per-token latency.

By Xinyu Lian, Walid Krichene, Beichen Huang, Masahiro Tanaka, Olatunji Ruwase, Li Zhang, Minjia Zhang

llms agents nlp efficiency benchmarks

arXiv Machine Learning

2d ago

RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models

The paper introduces RATIO, a framework for improving quantized reasoning models by identifying overthinking tokens and applying token-specific penalties. It uses Quantization-aware Reasoning Behavior Analysis to detect problematic tokens and Token-Specific Penalty Determination to assign penalties without extra training. Experiments show RATIO outperforms existing methods, boosting accuracy by up to 9.8 points and shortening chain-of-thought length by up to 51.3%.

By Chengzhu Bao, Xianglong Yan, Tianao Zhang, Jiaqi Chen, Shaoqiu Zhang, Yulun Zhang

llms efficiency

arXiv Machine Learning

Jun 2

Quantized Reasoning Models Think They Need to Think Longer, but They Do Not

arXiv:2606. 00206v1 Announce Type: new Abstract: Post-training quantization (PTQ) is widely used to deploy large language models efficiently, but its effect on reasoning models is not well understood.

By Sanae Lotfi, Polina Kirichenko, Steven Li, Zechun Liu

llms efficiency benchmarks

One email a morning, machine-written

Related stories

Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery

Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning

ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training

Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models

RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models

Quantized Reasoning Models Think They Need to Think Longer, but They Do Not