Accelerated Inference with Optimum and Transformers Pipelines

How we sped up transformer inference 100x for 🤗 API customers

Sparse Layers are Critical to Scaling Looped Language Models

arXiv:2605. 09165v2 Announce Type: replace Abstract: Looped language models repeat a set of transformer layers through depth, reducing memory costs and providing natural early-exit points at loop boundaries.

Recirculation

arXiv:2608. 17981v1 Announce Type: new Abstract: We describe an inference-time architectural enhancement for off-the-shelf foundation models that markedly reduces perplexity and boosts accuracy across generation and reasoning tasks.

ATLAS: Automated Approximation of Transformers for Efficient Homomorphic Inference in One Hour

arXiv:2607. 23478v1 Announce Type: cross Abstract: Fully homomorphic encryption (FHE) provides strong cryptographic guarantees for private inference, but deploying transformer models under FHE remains prohibitively expensive.

Up to 3.2x Faster Inference with LFM2.5-DSpark

Scaling up BERT-like model Inference on modern CPU - Part 2

FPTQuant: Function-Preserving Transforms for LLM Quantization

arXiv:2506. 04985v2 Announce Type: replace Abstract: Large language models (LLMs) require substantial compute, and thus energy, at inference time.

Large-scale empirical tuning and comparison of default optimizers for variational inference

arXiv:2606. 07841v1 Announce Type: cross Abstract: Black-box variational inference (BBVI) is a methodology for posterior approximation that relies on stochastic optimization.

c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimization

arXiv:2211. 14411v5 Announce Type: replace-cross Abstract: Hyperparameter optimization (HPO) is crucial for strong performance of deep learning algorithms and real-world applications often impose some constraints, such as on memory usage or latency, on top of the performance requirement.

Esoteric Language Models: A Family of Any-Order Diffusion LLMs

arXiv:2506. 01928v5 Announce Type: replace-cross Abstract: Diffusion-based language models offer a compelling alternative to autoregressive (AR) models by enabling parallel and controllable generation.

The Model Knows, the Decoder Finds: Future Value Guided Particle Power Sampling

arXiv:2605. 02427v3 Announce Type: replace Abstract: A recurring pattern in "reasoning without training" is that base LLMs already assign non-trivial probability mass to correct multi-step solutions; the bottleneck is locating these modes efficiently at inference time.

SNLP: Layer-Parallel Inference via Structured Newton Corrections

arXiv:2605. 17842v3 Announce Type: replace Abstract: Autoregressive language models execute Transformer layers sequentially, creating a latency bottleneck that is not removed by conventional tensor or pipeline parallelism.