Inference efficiency

Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.

3,245 stories · RSS feed

arXiv Machine Learning
Jul 27

Local-Global Geometric Insights for Graph Neural Networks via Entropic Curvature

arXiv:2607. 22381v1 Announce Type: new Abstract: Curvature notions on graphs, particularly Ollivier-Ricci and Forman, have emerged as powerful tools for addressing fundamental issues in Graph Neural Networks (GNNs) such as oversmoothing and oversquashing, but rely almost exclusively on local edge-level comparisons and therefore fail to certify how information actually propagates over long distances.

By Rachid Caich, Yassine Abbahaddou
Hugging Face Trending Papers
Jul 26

OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models

High-resolution image and video diffusion models, including SD3, FLUX, and recent video diffusion transformers, have substantially improved generative quality but remain expensive at inference time because they repeatedly evaluate attention-heavy denoisers over many sampling steps. We address this inefficiency by exploiting redundancy in intermediate diffusion features rather than changing model weights or retraining.

Hugging Face Trending Papers
Jul 26

PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis

Pathological diagnosis is inherently multi-scale, requiring the integration of global tissue architecture at low magnification with cellular morphology at higher magnification. However, existing pathology benchmarks and vision-language models (VLMs) are still largely developed under single-scale settings, limiting their ability to learn clinically meaningful multi-magnification reasoning.

Hugging Face Trending Papers
Jul 26

PathSelect: Sequential Token Selection for Whole Slide Pathology

Gigapixel Whole-Slide Images (WSIs) present a fundamental computational bottleneck for vision-language models (VLMs) due to extreme sequence lengths. Existing approaches predominantly rely on spatial sampling or training-free pruning, which risk diluting weak but informative signals, leading to the loss of critical diagnostic evidence due to the spatially diffuse nature of pathological cues.

Hugging Face Trending Papers
Jul 24

Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations

Post-training quantization (PTQ) has become a practical solution for deploying deep learning models on resource-constrained edge devices by compressing high-precision floating-point weights into low-precision representations without requiring retraining. Past research has demonstrated that quantization largely preserves classification accuracy; however, whether it also preserves the model's internal reasoning remains an open question.

Hugging Face Trending Papers
Jul 24

OrchNAS: Orchestrated Neural Architecture Search Service for Personalised Federated Edge Intelligence

We propose OrchNAS, an energy-aware, personalised, federated edge intelligence framework that leverages a Neural Architecture Search Service to automatically design service-adaptive models for heterogeneous edge environments. The framework orchestrates the architecture search process on a server-side NAS service, enabling edge services to derive personalised architectures under device-level energy, computation, and memory constraints.

arXiv AI
Jul 24

EvoSQL: Memory-Augmented Critic-Generator Co-Evolution for Text-to-SQL

arXiv:2607. 20489v1 Announce Type: new Abstract: Text-to-SQL has advanced rapidly with large language models, but complex database queries still require reasoning beyond one-shot generation, including multi-step decomposition, execution-based diagnosis, and targeted correction.

By Jiawei Zhou, Jianwei Wang, Chenyu Zhou, Chaojian Shi, Ming Dong, Kai Wang
arXiv Machine Learning
Jul 24

KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers

arXiv:2607. 21446v1 Announce Type: new Abstract: Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-bit formats cannot represent.

By Yann Bouquet, Alireza Khodamoradi, Kristof Denolf, Mathieu Salzmann