arXiv AI
X-Stage: Modeling Post-Issue Backpressure in GPU Communication--Computation Fusion
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 28
X-Stage: An Overlooked Pipeline Stage for Communication-Computation Overlap in DiT Inference
arXiv:2607. 23264v1 Announce Type: cross Abstract: Fine-grained, device-initiated communication lets persistent GPU kernels in distributed diffusion transformer (DiT) inference issue remote stores and overlap data movement with Tensor Core computation.
arXiv AI
Jul 7
Fine-Grained Computation Offload for Off-the-Shelf Servers in Tens of Lines
arXiv:2607. 02630v1 Announce Type: cross Abstract: Hardware accelerators now sit on the critical path of online serving.
arXiv Machine Learning
Jul 28
FusionML: Prefill, Not Decode - Mechanism and Boundaries of CPU+GPU Co-Execution on Unified-Memory Apple Silicon
arXiv:2607. 22785v1 Announce Type: cross Abstract: Apple-Silicon SoCs share CPU, GPU, and Neural Engine over one unified memory system, raising the question of whether transformer inference can be accelerated by splitting single operators across units.
arXiv AI
Aug 20
Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets
The paper presents a method for distributing large language model inference across multiple Intel AI PCs by splitting the model into pipeline shards, each pre‑compiled into an OpenVINO graph. Three key techniques—beam_idx Gather to enable GPU optimizations, speculative decoding on stateful models, and interleaved micro‑batching—allow a two‑node Llama 3.1 8B INT4 pipeline to serve two users at 1.79× the throughput of a single‑node model, while a four‑node deployment can run a 70B model that no single PC can hold. The authors provide code, benchmark logs, and reproduction scripts on GitHub.
arXiv Machine Learning
Aug 7
Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening
arXiv:2608. 05944v1 Announce Type: cross Abstract: We report operational experience full-fine-tuning a 32.
arXiv AI
Jul 3
Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving
arXiv:2607. 02043v1 Announce Type: cross Abstract: Disaggregated LLM serving runs prefill and decode on separate GPU pools to keep the two phases from interfering.