arXiv AI July 7, 2026 By Ferran Agullo, Joan Oliveras, Chen Wang, Alberto Gutierrez-Torre, Olivier Tardieu, Alaa Youssef, Jordi Torres, Josep Ll. Berral

Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving

Read the original on arXiv AI →

arXiv:2602. 24044v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 2

Lodestar: An Online-Learning LLM Inference Router

arXiv:2606. 00946v1 Announce Type: cross Abstract: Efficiently serving large language model (LLM) inference tasks is crucial both for user-perceived latency such as time-to-first-token (TTFT) and for GPU utilization.

By Gangmuk Lim, Wanyu Zhao, Brighten Godfrey, Jiaxin Shan, Le Xu, Liguang Xie

llms benchmarks

arXiv AI

Sep 10

A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

arXiv:2609.08307v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires bala...

By Maysam Khatib, Moysis Symeonides, Demetris Trihinas, George Pallis, Marios D. Dikaiakos

llms efficiency benchmarks

arXiv Machine Learning

Sep 15

BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference

arXiv:2609.13592v1 Announce Type: cross Abstract: GPU memory bandwidth and capacity limit throughput in large language model (LLM) inference. The GPU memory system consists of a primary tier of high-...

By Anish Saxena, Jae Hyung Ju, Hritvik Taneja, Po-An Tsai, Aamer Jaleel, Christos Kozyrakis, Moinuddin Qureshi

llms efficiency

arXiv Machine Learning

4d ago

Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging

arXiv:2609.38090v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models are a compelling architecture for scaling model capacity, making them especially attractive for deployment on resource-...

By Sanjali Yadav, Bahar Asgari

efficiency benchmarks

arXiv AI

Jun 30

KernelSight-LM: A Kernel-Level LLM Inference Simulator

arXiv:2606. 28565v1 Announce Type: cross Abstract: As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to meet cost and latency targets.

By Xiteng Yao, Taeho Kim, Hengzhi Pei, Xinle Liu, Kyle Ulrich, Leonard Lausen, Ashish Khetan, Xiang Song, George Karypis, Martin Herbordt

llms efficiency benchmarks

arXiv AI

Jun 9

Meeting SLOs, Slashing Hours: Automated Enterprise LLM Optimization with OptiKIT

arXiv:2601. 20408v2 Announce Type: replace-cross Abstract: Enterprise LLM deployment faces a critical scalability challenge: organizations must optimize models systematically to scale AI initiatives within constrained compute budgets, yet the specialized expertise required for manual optimization remains a niche and scarce skillset.

By Nicholas Santavas, Kareem Eissa, Patrycja Cieplicka, Piotr Florek, Matteo Nulli, Stefan Vasilev, Seyyed Hadi Hashemi, Antonios Gasteratos, Shahram Khadivi

llms agents efficiency

One email a morning, machine-written

Related stories

Lodestar: An Online-Learning LLM Inference Router

A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference

Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging

KernelSight-LM: A Kernel-Level LLM Inference Simulator

Meeting SLOs, Slashing Hours: Automated Enterprise LLM Optimization with OptiKIT