arXiv:2607. 02630v1 Announce Type: cross Abstract: Hardware accelerators now sit on the critical path of online serving.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2607. 02043v1 Announce Type: cross Abstract: Disaggregated LLM serving runs prefill and decode on separate GPU pools to keep the two phases from interfering.
The paper introduces Decode‑Latency Feedback Prefill (DLFP), a model‑free controller that adjusts prefill chunk sizes during concurrent autoregressive inference to reduce interference between new and ongoing requests. Implemented in vLLM, DLFP achieves significant reductions in P99 inter‑token latency on Qwen3‑0.6B while maintaining output correctness and SLO compliance, though it fails to generalize to larger models or multi‑GPU setups. The study highlights the limits of this approach and suggests the need for a completion‑timed controller for broader applicability.
arXiv:2512. 10236v2 Announce Type: replace-cross Abstract: Modern ML workloads demand distributing training and inference across multiple GPUs.
arXiv:2609.37626v1 Announce Type: cross Abstract: No single way of parallelizing attention serves large language models well under all loads. Low concurrency favors tensor parallelism, many independe...
We report operational experience full-fine-tuning a 32. 76B-parameter dense model (Qwen3-32B) on 16 x NVIDIA B300 (two nodes, FSDP / ZeRO-3) -- among the first published field accounts on this accelerator.
arXiv:2608. 05944v1 Announce Type: cross Abstract: We report operational experience full-fine-tuning a 32.