arXiv AI

Ekka: Automated Diagnosis of Silent Errors in LLM Inference

arXiv:2606. 04594v1 Announce Type: cross Abstract: LLM serving frameworks are quickly evolving with a complex software stack and a vast number of optimizations.

arXiv Machine Learning
Aug 4

TELLER: Non-intrusive Cross-Layer Root-Cause Analysis for LLM Inference

arXiv:2608. 01975v1 Announce Type: cross Abstract: Large language model (LLM) inference has evolved from an offline workload into a continuously operated software service, yet root-cause analysis remains difficult because a single request spans the inference engine, Python/C++ backend, host CUDA APIs, GPU kernels, and distributed communication.

By Ruilin Xu, Junyi Li, Pengfei Chen, Zongxuan Xie