Bayesian inference provides a principled foundation for reasoning under uncertainty, but its computational cost hinders deployment on resource-constrained edge devices. In this paper, we present a hardware-oriented methodology for accelerating discrete Bayesian inference on commercial off-the-shelf embedded GPUs.
arXiv:2603. 29002v3 Announce Type: replace-cross Abstract: Modern large language models (LLMs) increasingly depends on efficient long-context processing and generation mechanisms, including sparse attention, retrieval-augmented generation (RAG), and compressed contextual memory, to support complex reasoning.
By Zifan He, Rui Ma, Yizhou Sun, Jason Cong
arXiv:2607. 08786v1 Announce Type: cross Abstract: With the growing deployment of large language models (LLMs), LLM inference cost has become a key challenge.
By Tao Lu, Haoyu Wang, Zonghui Wang, Keshen Xiang, Jiaheng Zhang, Wenzhi Chen
arXiv:2607. 21985v1 Announce Type: cross Abstract: The increasing deployment of large language models (LLMs) has magnified the computational and memory bottlenecks of autoregressive decoding, where low compute intensity and bandwidth-bound kernels dominate inference cost.
By Jinhyeok Kim, Yejoon Lee, Jaeyoung Do
arXiv:2501.10375v3 Announce Type: replace-cross
Abstract: Mixture-of-Experts (MoE) models, though highly effective for various machine learning tasks, face significant deployment challenges on memory...
By Yujie Zhang, Shivam Aggarwal, Tulika Mitra
The paper introduces a partition-aware scheduling framework for mobile inference on heterogeneous platforms that combines mobile GPUs and multiple CPU core clusters. It jointly optimizes operator partitioning, device assignment, and execution order for static DAGs of operators, such as those in CNNs or vision transformers. An online iterative search approach decomposes large DAGs into stages, targets critical operators, and uses latency predictors to avoid exhaustive profiling, achieving near‑optimal latency with minimal scheduling overhead.
By Zhuojin Li, Marco Paolieri, Leana Golubchik