arXiv AI
Jul 7
Fine-Grained Computation Offload for Off-the-Shelf Servers in Tens of Lines
arXiv:2607. 02630v1 Announce Type: cross Abstract: Hardware accelerators now sit on the critical path of online serving.
MADAR: An Address-Free Processor
arXiv:2606. 15535v1 Announce Type: cross Abstract: In a modern processor, computing is the cheap part.
Related stories
arXiv AI
5d ago
TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes
arXiv:2608. 13057v1 Announce Type: cross Abstract: In expert-parallel (EP) MoE serving, every layer synchronizes at the slowest GPU.
arXiv AI
Jul 21
Memory in the Loop: In-Process Retrieval as Extended Working Memory for Language Agents
arXiv:2607. 05690v2 Announce Type: replace Abstract: Language agents run a loop - observe, reason, act - but the memory they reason over sits outside it: a store queried at most once per turn.
arXiv Machine Learning
Jul 7
Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
arXiv:2607. 04206v1 Announce Type: cross Abstract: Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation.
arXiv AI
Jul 8
Memory in the Loop: In-Process Retrieval as ExtendedWorking Memory for Language Agents
arXiv:2607. 05690v1 Announce Type: new Abstract: Language agents run a loop - observe, reason, act - but the memory they reason over sits outside it: a store queried at most once per turn.
Hugging Face Trending Papers
Jul 5
Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
Diffusion language models (dLLMs) generate text by iteratively denoising a masked response and can commit multiple output positions per model invocation. Their bidirectional attention prevents exact autoregressive-style KV caching, since committing one position shifts the KV activations of all others.
arXiv Machine Learning
Jun 8
Hermes: Accelerating Long-Latency Load Requests via Perceptron-Based Off-Chip Load Prediction
arXiv:2209. 00188v4 Announce Type: replace-cross Abstract: Long-latency load requests continue to limit the performance of high-performance processors.
arXiv AI
Jun 9
AGENTSERVESIM: A Hardware-aware Simulator for Multi-Turn LLM Agent Serving
arXiv:2606. 09613v1 Announce Type: cross Abstract: Multi-turn LLM agents interleave model calls with external tool invocations, shifting serving from stateless request processing to stateful program execution.
arXiv AI
Jul 7
Elastic Gang: Per-Token Membership Change for a Hard-Barriered LLM Inference Gang Co-Scheduled with OS Processes
arXiv:2607. 04668v1 Announce Type: cross Abstract: On-device LLM decoding is a hard-barriered CPU-SIMD computation that wants every core for milliseconds per token, while the rest of the OS wants those same cores continuously.
arXiv Machine Learning
Aug 7
Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening
arXiv:2608. 05944v1 Announce Type: cross Abstract: We report operational experience full-fine-tuning a 32.
arXiv AI
Jul 8
Think Before You Grid-Search: Floor-First Triage for LLM Serving
arXiv:2607. 05876v1 Announce Type: cross Abstract: LLM serving optimization typically benchmarks many configurations and reaches for heavy profilers when latency targets are missed.
arXiv Machine Learning
Jun 2
Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving
arXiv:2606. 01839v1 Announce Type: cross Abstract: LLM-based agents resolve a user task through many turns of dependent inference and tool calls, producing a workload whose total cost is unknown when the task arrives.