Hugging Face Trending Papers
ROCm InferaSim: Cut LLM Serving Trial Costs and Deploy the Best Configuration
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Jul 17
Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
arXiv:2607. 14541v1 Announce Type: new Abstract: Existing GPU kernel generation benchmarks draw problems from synthetic or curated sources that diverge from deployed workloads.
arXiv AI
Jun 30
KernelSight-LM: A Kernel-Level LLM Inference Simulator
arXiv:2606. 28565v1 Announce Type: cross Abstract: As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to meet cost and latency targets.
Hugging Face Trending Papers
Oct 14
Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training
Configuring distributed LLM training across hundreds of GPUs requires jointly optimizing tensor, pipeline, expert, context, and data parallelism alongside micro-batch size, recomputation, sharding, an...
arXiv AI
Sep 16
Calibrate, Then Route: A Measured Study of Learned Request Routing for Disaggregated LLM Serving
The paper evaluates a learned request‑routing policy for disaggregated large‑language‑model serving, where compute‑heavy prefill and memory‑heavy decode stages run on separate GPU pools. Using a discrete‑event simulator and real NVIDIA A40 GPUs, the calibrated router—leveraging prompt length, predicted output length, KV‑cache pressure, and SLO class—outperforms round‑robin, least‑loaded, and length‑based heuristics, achieving the highest mean goodput (0.864) and lowest variance across three mixed, bursty arrival traces. Hardware calibration proves critical, providing a 4.5‑point goodput boost and roughly 40 % of the tail‑latency advantage, and the learned router can match round‑robin performance with one fewer GPU in certain scenarios.
Hugging Face Trending Papers
Jul 2
WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs
Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption. Optimizing these deployments requires matching specific LLMs to the most efficient GPUs, but operators currently lack the tools to do so without exhaustively profiling each combination.
arXiv Machine Learning
Aug 4
Meganeura: Portable GPU Training and Inference through Vulkan and Metal
arXiv:2608. 01563v1 Announce Type: new Abstract: Training and deployed inference often cross export, conversion, and platform-specific runtime boundaries.