Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

3,534 stories · RSS feed

arXiv AI
Jun 8

FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models

arXiv:2606. 06547v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) refine tokens iteratively but commit them irreversibly, leading to a "stability lag" where early decisions remain fragile even after being written.

By Haoyu Huang, Linlin Yang, Sheng Xu, Boyu Liu, Guodong Guo, Zhongqian Fu, Hang Zhou, Baochang Zhang
arXiv Machine Learning
Jun 8

Deployed trusted-node quantum key distribution over 300 km with a multi-core fiber access link

arXiv:2606. 06107v1 Announce Type: cross Abstract: Quantum key distribution (QKD) is increasingly considered for deployment in realistic communication networks, where long distances, heterogeneous fiber infrastructure, and coexistence with classical traffic present substantial challenges.

By Martin Clason, Joakim Argillander, Didrik Bergstr\"om, Daniel Spegel-Lexne, Giulio Foletto, Ashraf El Hassan, Mohamed Bourennane, Onur G\"unl\"u, Katia Gallo, Rui Lin, Guilherme B. Xavier
Hugging Face Trending Papers
Jun 8

From Hazard Functions to Language Space: Cox-Supervised Distillation of Survival Risk into a Large Language Model

We investigate whether information about time-to-event risk estimated by a Cox proportional hazards model can be transferred into a generative large language model. We propose a text-based survival modelling pipeline in which structured clinical covariates are converted into text prompts and a Qwen-based large language model is fine-tuned to generate patient-specific survival risk using Cox model predictions as a training target.

Hugging Face Trending Papers
Jun 8

FAME: Forecastability-Aware Mixture of Experts for Heterogeneous Time Series Forecasting

Large-scale retail and industrial forecasting systems contain many heterogeneous time series whose lifecycle, sparsity, volatility, seasonality, spectral patterns, and contextual sensitivity differ substantially. A single forecasting model rarely performs well across all regimes, while dense ensembles increase inference cost and provide limited insight into expert suitability.

arXiv AI
Jun 6

SAGE: Scalable AI Governance & Evaluation

arXiv:2602. 07840v3 Announce Type: replace-cross Abstract: Evaluating relevance in large-scale search systems is fundamentally constrained by the governance gap between nuanced, resource-constrained human oversight and the high-throughput requirements of production systems.

By Benjamin Le, Xueying Lu, Nick Stern, Wenqiong Liu, Igor Lapchuk, Xiang Li, Baofen Zheng, Kevin Rosenberg, Jiewen Huang, Zhe Zhang, Abraham Cabangbang, Satej Milind Wagle, Jianqiang Shen, Raghavan Muthuregunathan, Abhinav Gupta, Mathew Teoh, Andrew Kirk, Thomas Kwan, Jingwei Wu, Wenjing Zhang
arXiv AI
Jun 6

Towards Generalization of Block Attention via Automatic Segmentation and Block Distillation

arXiv:2605. 15913v4 Announce Type: replace-cross Abstract: Block attention, which processes the input as separate blocks that cannot attend to one another, offers significant potential to improve KV cache reuse in long-context scenarios such as Retrieval-Augmented Generation (RAG).

By Shuaiyi Li, Zhisong Zhang, Yan Wang, Lei Zhu, Dongyang Ma, Chenlong Deng, Yang Deng, Wai Lam
arXiv AI
Jun 6

Evidence Over Plans: Online Trajectory Verification for Skill Distillation

arXiv:2605. 09192v2 Announce Type: replace Abstract: Agent skills can remarkably improve task success rates by using human-written procedural documents, but their quality is difficult to assess without environment-grounded verification.

By Yang Zhou, Zihan Dong, Zhenting Wang, Can Jin, Shiyu Zhao, Bangwei Guo, Difei Gu, Linjun Zhang, Mu Zhou, Dimitris N. Metaxas
arXiv AI
Jun 6

Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Driving

arXiv:2601. 21288v2 Announce Type: replace Abstract: Autonomous driving is an important and safety-critical task, and recent advances in LLMs/VLMs have opened new possibilities for reasoning and planning in this domain.

By Weitong Lian, Zecong Tang, Haoran Li, Tianjian Gao, Yifei Wang, Zixu Wang, Lingyi Meng, Tengju Ru, Zhejun Cui, Yichen Zhu, Hangshuo Cao, Qi Kang, Tianxing Chen, Kaixuan Wang, Yu Zhang