arXiv AI

Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation

arXiv:2606. 05682v2 Announce Type: replace Abstract: Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments.

arXiv Machine Learning

Jun 5

Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillatio

arXiv:2606. 05682v1 Announce Type: cross Abstract: Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments.

By Fangbo Tu, Junhua Zhao, Chi Liu, Xin Chen, Haifeng Wu, Jian Wan, Srinivasan Manoharan

llms reinforcement-learning efficiency safety

arXiv AI

Jun 29

Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs

arXiv:2601. 22709v5 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) achieve strong multimodal performance but are costly to deploy, and post-training quantization often causes significant accuracy loss.

By Yanlong Chen, Amirhossein Habibian, Luca Benini, Yawei Li

llms efficiency multimodal benchmarks safety

arXiv AI

Jun 4

Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

arXiv:2606. 04238v1 Announce Type: cross Abstract: Aggressive weight quantization to 2-bit precision offers substantial throughput and memory gains for large language model (LLM) inference, but typically incurs severe accuracy degradation.

By Devleena Das, Rajeev Patwari, Elliott Delaye, Ashish Sirasao

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 29

Stable FP4 Training via Transposition-Invariant Block Quantization

arXiv:2607. 24953v1 Announce Type: cross Abstract: Reducing training precision is a key lever for improving the e ciency of large language model (LLM) training, but pushing beyond FP8 to 4-bit oating point (FP4) remains challenging due to instability during optimization.

By Mehdi Rahimifar, Amin Darabi, Mehran Taghian Jazi, Xing Huang, Yao Wang, Zhijun Tu, Yufei Cui, Yunke Peng, Hongliang Li

llms efficiency

arXiv AI

22h ago

Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting

arXiv:2604. 15794v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have achieved remarkable success, underpinning diverse AI applications.

By Chi Liu, Xin Chen, Xu Zhou, Fangbo Tu, Srinivasan Manoharan

llms fine-tuning efficiency safety

arXiv AI

Jun 3

Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancement

arXiv:2412. 01282v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) bring powerful understanding and reasoning capabilities to multimodal tasks.

By Qianhan Feng, Wenshuo Li, Tong Lin, Xinghao Chen

llms rag efficiency multimodal benchmarks safety

arXiv Machine Learning

Jul 20

QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides

arXiv:2607. 15810v1 Announce Type: new Abstract: Rollout generation is a major bottleneck in Reinforcement Learning (RL) for Mixture-of-Experts (MoE) Large Language Models, motivating low-precision rollout acceleration such as FP8.

By Zhengyang Zhuge, Hao Yu, Xin Wang, Zheng Li, Yizhong Cao, Dayiheng Liu, Jianwei Zhang

llms reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Jun 16

ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training

arXiv:2606. 15682v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) achieve strong problem-solving through long chain-of-thought, but their deployment is constrained by the high cost of full-precision inference and growing KV cache footprints.

By Janghwan Lee, Sihwa Lee, Jinseok Kim, Yongjik Kim, Jieun Lim, Jinwook Oh, Jungwook Choi

fine-tuning efficiency data-engineering

arXiv Machine Learning

Jul 2

Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization

arXiv:2607. 00908v1 Announce Type: new Abstract: Mixed-precision quantization (MPQ) has become a key technique for deploying large language models under stringent memory and compute constraints.

By Fei Wang, Chao Xue, Taoran Liu, Li Shen, Ye Liu, ChangXing Ding

llms efficiency safety

arXiv Machine Learning

Jun 18

Contextualizing Biological Language Models across Modalities via Logit-Space Contrastive Alignment

arXiv:2606. 18703v1 Announce Type: new Abstract: Pretrained biological language models expose per-token probability distributions through masked-token prediction, providing the likelihood interface central to sequence design, variant scoring, and mechanistic interpretation.

By Yanjun Shao, Yundi Chen, Yashvi Patel, Aurelien Pelissier, Mar\'ia Rodr\'iguez Mart\'inez

llms rag nlp multimodal benchmarks safety

arXiv Machine Learning

Jun 26

The Geometry of Updates: Fisher Alignment at Vocabulary Scale

arXiv:2606. 27242v1 Announce Type: new Abstract: Training-free source selection for LLM families with shared vocabularies arises in scientific string domains such as SMILES, protein, and genomic sequences, where candidate corpora share a tokenizer but differ in prediction targets.

By John Sweeney

llms rag nlp safety

arXiv Machine Learning

Jun 26

What Survives When You Compress a Recursive Reasoner for the Edge?

arXiv:2606. 26488v1 Announce Type: new Abstract: Recursive reasoning models can solve complex structured tasks with only a few million parameters by repeatedly updating a latent state.

By Pearse Jim, Steven Kolawole, Opegbemi Matthias Busoye, Glory Bagai, Virginia Smith

rag efficiency

Related stories