The paper examines the commonly used structure‑disjoint split for evaluating molecular machine learning models, which relies on single‑linkage clustering with the Maximum Common Edge Subgraph (MCES) distance at a threshold of 10. The authors show that this criterion, originally calibrated on 29,000 molecules, has never been tested on the current public libraries of 227,690 molecules due to its NP‑hard complexity. They propose an optimized algorithm that reduces the number of required integer programs from $2.6 imes10^{10}$ to $1.15 imes10^{7}$, enabling lossless partitioning at scale, and reveal that the resulting graph percolates—over 95 % of the library lies in a single component, making held‑out compounds systematically larger and more complex than training ones.
"whyItMatters":"The study highlights that large‑library benchmarks using the traditional split conflate structural novelty with extrapolation to larger, more complex chemistry, potentially misleading assessments of model generalization."
By Tomo Oga, Devesh Shah, Cailum M. K. Stienstra, Gabriel Asher, Antonio H. de O. Fonseca, Niall O'Connor, Michael Widrich
arXiv:2606. 24722v1 Announce Type: new Abstract: Frontier AI training is increasingly shaped by access to dense, centrally controlled accelerator clusters.
By Peter Toth
The paper presents a method for distributing large language model inference across multiple Intel AI PCs by splitting the model into pipeline shards, each pre‑compiled into an OpenVINO graph. Three key techniques—beam_idx Gather to enable GPU optimizations, speculative decoding on stateful models, and interleaved micro‑batching—allow a two‑node Llama 3.1 8B INT4 pipeline to serve two users at 1.79× the throughput of a single‑node model, while a four‑node deployment can run a 70B model that no single PC can hold. The authors provide code, benchmark logs, and reproduction scripts on GitHub.
By Tate Berenbaum, Muthaiah Venkatachalam
The paper introduces Group-Query Latent Attention (GQLA), a variant of Multi-head Latent Attention that shares group‑sharded up‑projections to enable a joint latent representation and a shardable expanded cache. GQLA supports both MQA‑absorb and GQA execution using the same weights and existing kernels, and roofline analysis shows how absorption trades traffic for computation while GQA shards KV groups. Experiments on DeepSeek‑V3.1‑Base and other models demonstrate that GQLA achieves comparable training losses to MLA, maintains high accuracy after checkpoint conversion, and improves output throughput by up to 35% on eight H20‑3e GPUs, while also allowing larger batch sizes due to a smaller per‑rank cache.
By Fanxu Meng, Zhixin Pan
The paper investigates how to optimally partition streaming inference models into megakernels and conventional graph components. It introduces a framework that predicts the best megakernel composition without full end‑to‑end runs, showing that multiple megakernels or a fully conventional graph often outperform a single megakernel across various models and batch sizes. The study defines four model categories based on megakernel stability and demonstrates significant latency and throughput gains on an H100 NVL GPU using a three‑megakernel design.
By Aditya Chatterjee, Anand Joseph X M, Sanjay Singh Chauhan
The paper introduces ERASE, a training schedule that detaches subgraphs in a neural network to launch their backward passes early on separate CUDA streams, thereby overlapping backward and forward computations. By applying this technique to a lightweight transformer and a large‑scale click‑through‑rate model, the authors demonstrate up to a 9.51% increase in training throughput while maintaining performance metrics close to the baseline.
By Ergan Shang, Flavio Sales Truzzi
We address a fundamental question: How far should kernel fusion extend and which boundaries are worth preserving? Unlike the general convention of using one monolith megakernel at lower batch sizes or...
arXiv:2610.19822v1 Announce Type: cross
Abstract: LLM agents are evolving from a single model calling tools to planners that orchestrate multiple expert models whose capabilities live in distinct wei...
By Daijin Hu
The paper investigates the performance impact of using a remote KV cache tier for long‑context language model serving. In a disaggregated deployment of GLM‑5.3 on 8xB300 nodes, enabling the remote tier initially increased the 16k‑token p99 latency from 3.96 s to 17.3 s, and synchronous loading caused severe queue instability. By sharding the KV store across context‑parallel ranks, partitioning the remote tier, adding asynchronous loading, and fixing client bugs, the authors reduced the p99 latency from 83.32 s to 0.67 s and increased prefill throughput by 14–34 %.
whyItMatters":"The study demonstrates that careful sharding and asynchronous handling of remote KV caches can dramatically improve latency and throughput in large‑scale LLM serving systems."
By Muhammad Ibrahim
arXiv:2607. 13332v1 Announce Type: new Abstract: Training large language models at the multi-billion to trillion parameter scale is confined to datacenters, where data-parallel (DP) and model-parallel (MP) techniques presume homogeneous accelerators, high-speed interconnects, and a single orchestrating entity.
By Gil Avraham, Violetta Shevchenko, Hadi Mohaghegh Dolatabadi, Karol Pajak, James Snewin, Harry Xi, Rodney O'Donnell, Thalaiyasingam Ajanthan, Sameera Ramasinghe, Chamin Hewa Koneputugodage, Shamane Siriwardhana, Alexander Long
The paper introduces Tessera, a distributed runtime that enables equivariant message‑passing Transformers to simulate molecular dynamics at ab initio accuracy for tens of millions of atoms. By sorting atoms along a space‑filling curve and maintaining consistent workload shapes, Tessera achieves near‑linear scaling on 512 GPUs, sustaining 30.7 million atoms with high weak‑scaling efficiency and low energy drift in large‑scale simulations.
By Allex Kan, Ji Zhang, Hao Yin, Hao Li, Owen Fu, Xunxing Chen, Lin Huang, Zion Wang, Lucas Li, Jiajun Cheng, Arthur Jiang, Junping Zhao, Jia Zhang
Zephon is a data loader designed for foundation model training that ensures deterministic ordering of training data batches even when GPU resources change, checkpoints are resumed, or different processing backends are used. It handles online, stateful pipelines—where tokenization, packing, and mixing of samples create complex n‑to‑m transformations—by partitioning the data stream into topology‑independent lanes, serializing ordering decisions, and checkpointing only bounded in‑flight state. Experiments on text and vision‑language tasks show Zephon delivers competitive throughput while offering guarantees that existing loaders lack for such pipelines.
By Maximilian B\"other, Josh Wills, Ties Robroek, Sonnet Xu, Paul Burstein, Daniel Zayas, Cody Blakeney, Siddharth Joshi, Haoli Yin, Rishabh Adiga, Haakon Mongstad, Luke Merrick, Pratyush Maini, Ari Morcos, Matthew Leavitt, Ana Klimovic, Bogdan Gaza