Hugging Face Trending Papers August 17, 2026

Pallas: A Proactive KV Cache Migration Framework for LLM Inference in AI-RAN

Read the original on Hugging Face Trending Papers →

AI-RAN brings large language model (LLM) serving close to mobile users, but cellular handover can separate an active request from its inference state: the user attaches to a target base station (gNB) while the large and growing key-value (KV) cache remains at the source. Retaining inference at the source preserves service continuity but persistently increases inter-token latency (ITL), whereas recovering the state at the target restores serving locality but requires KV-cache transfer, recomputation, or a combination of both only after handover, directly prolonging service interruption time (SIT).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Aug 18

Pallas: A Proactive KV Cache Migration Framework for LLM Inference in AI-RAN

arXiv:2608. 16477v1 Announce Type: new Abstract: AI-RAN brings large language model (LLM) serving close to mobile users, but cellular handover can separate an active request from its inference state: the user attaches to a target base station (gNB) while the large and growing key-value (KV) cache remains at the source.

By Tianhang Ding, Jianchun Liu, Hongli Xu

llms efficiency

arXiv AI

Sep 1

A Universal Context-Reuse Layer for Cross-Model KV Sharing

arXiv:2608.30963v1 Announce Type: cross Abstract: Modern large language model (LLM) serving systems increasingly operate over repeated or shared context, yet each model typically performs its own pre...

By Yi Li, Dongming Jiang, Yi Zhao, Bingzhe Li

llms agents nlp

arXiv AI

Jul 10

SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling

arXiv:2607. 08565v1 Announce Type: cross Abstract: LLM scheduling is critical to serving, yet it remains unclear how well existing designs fit agentic serving--with LLM requests issued by agents instead of humans.

By Jiahao Wang, Kaizhan Lin, Kaixi Zhang, Jinbo Han, Xingda Wei, Sijie Shen, Chenguang Fang, Wenyuan Yu, Rong Chen, Haibo Chen

llms agents benchmarks

Hugging Face Trending Papers

Sep 15

Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs

A common small-model deployment runs one shared backbone with several LoRA specialists that answer over the same context. Serving them naively re-prefills that shared context once per specialist. We s...

fine-tuning efficiency

arXiv AI

Jul 21

Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs

arXiv:2607. 17181v1 Announce Type: cross Abstract: Serverless multi-model LLM systems multiplex popularity-skewed model catalogs over shared GPU pools, yet typically schedule each request independently.

By Utopia Meng, Unicornt Zhao, Derek Li, Goalen Gao, Frank Du

llms agents

Hugging Face Trending Papers

Jul 5

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attention prevents exact autoregressive-style KV caching, since committing one position shifts the KV activations of all others.

llms diffusion

One email a morning, machine-written

Related stories

Pallas: A Proactive KV Cache Migration Framework for LLM Inference in AI-RAN

A Universal Context-Reuse Layer for Cross-Model KV Sharing

SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling

Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs

Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack