🚀 Accelerating LLM Inference with TGI on Intel Gaudi

Optimum-NVIDIA Unlocking blazingly fast LLM inference in just 1 line of code

Prefill/Decode-Aware Evaluation of LLM Inference on Emerging AI Accelerators

arXiv:2606. 17104v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed in latency- and cost-sensitive settings, inference efficiency has become a central systems challenge.

Tile-Level Activation Overlap for Efficient LLM Inference

arXiv:2607. 02521v1 Announce Type: cross Abstract: SwiGLU is the dominant MLP activation in modern large language models, yet its intermediate tensor materialization costs 9-37% of MLP execution time.

Understanding and Coding the KV Cache in LLMs from Scratch

KV caches are one of the most critical techniques for efficient inference in LLMs in production.

FastTPS: An Optimized Method for LLM Token Phase for AI accelerators

arXiv:2607. 11211v1 Announce Type: new Abstract: The popularity of large language models (LLMs) escalates an ongoing demand for effective inference.

Accelerating Protein Language Model ProtST on Intel Gaudi 2