The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2608. 07911v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models have outgrown accelerator memory, and offloading expert weights to host memory is now standard.
The paper investigates whether training Mixture-of-Experts (MoE) routers can improve memory‑bandwidth locality on consumer GPUs. Using a new zero‑surgery telemetry tool, the authors measure that a large Qwen3‑235B model is bottlenecked by disk‑based expert access, and that an LRU cache can serve a majority of requests. They pre‑register experiments training 137 M‑parameter MoE models with locality‑aware losses, finding that while cache misses can drop up to 60 % (99 % static‑pin hit rate), every configuration fails to meet a strict 1 % perplexity threshold, indicating a tight coupling between cache efficiency and model quality.
arXiv:2608. 15089v1 Announce Type: new Abstract: Long-horizon agents can fail even when their underlying models can solve the constituent steps.
arXiv:2608. 05784v1 Announce Type: new Abstract: Computer-use agents pay full frontier inference to re-derive routines their user has already performed, because an agent's memory today records what the user said, not what the user did.
arXiv:2606. 28876v3 Announce Type: replace-cross Abstract: Proposal.
Serving a 235B-parameter Mixture-of-Experts (MoE) model on a single 8 GB GPU is bottlenecked not by compute but by memory bandwidth: decode must stream each token's active experts from whichever tier holds them, and on consumer hardware most experts sit on an SSD far slower than RAM. We quantify this bandwidth wall on Qwen3-235B (Q4_K_M, 134 GB): measured decode is 0.