arXiv Machine Learning July 16, 2026 By Gil Avraham, Violetta Shevchenko, Hadi Mohaghegh Dolatabadi, Karol Pajak, James Snewin, Harry Xi, Rodney O'Donnell, Thalaiyasingam Ajanthan, Sameera Ramasinghe, Chamin Hewa Koneputugodage, Shamane Siriwardhana, Alexander Long

Agora: Collective and Permissionless Internet-Scale Pretraining of Large Language Models

Read the original on arXiv Machine Learning →

arXiv:2607. 13332v1 Announce Type: new Abstract: Training large language models at the multi-billion to trillion parameter scale is confined to datacenters, where data-parallel (DP) and model-parallel (MP) techniques presume homogeneous accelerators, high-speed interconnects, and a single orchestrating entity.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Jun 18

FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs

arXiv:2606. 19025v1 Announce Type: cross Abstract: Pre-training Large Language Models (LLMs) typically demands large-scale infrastructure with tightly coupled hardware accelerators.

By Lorenzo Sani, Zeyu Cao, Meghdad Kurmanji, Alex Iacob, Andrej Jovanovic, Yan Gao, Wanru Zhao, Nicholas D. Lane

llms efficiency benchmarks

arXiv AI

Jun 2

Lodestar: An Online-Learning LLM Inference Router

arXiv:2606. 00946v1 Announce Type: cross Abstract: Efficiently serving large language model (LLM) inference tasks is crucial both for user-perceived latency such as time-to-first-token (TTFT) and for GPU utilization.

By Gangmuk Lim, Wanyu Zhao, Brighten Godfrey, Jiaxin Shan, Le Xu, Liguang Xie

llms benchmarks

arXiv Machine Learning

Jun 2

Don't Let a Few Network Failures Slow the Entire AllReduce

arXiv:2606. 01680v1 Announce Type: cross Abstract: Network failures are among the most frequent hardware faults in large-scale GPU clusters and a leading cause of training-job interruptions.

By Peiqing Chen, Jiedong Jiang, Nengneng Yu, Yuefeng Wang, Sixian Xiong, Wei Wang, Zaoxing Liu

benchmarks

arXiv AI

Jun 24

Decentralised AI Training and Inference with BlockTrain

arXiv:2606. 24722v1 Announce Type: new Abstract: Frontier AI training is increasingly shaped by access to dense, centrally controlled accelerator clusters.

By Peter Toth

llms

arXiv AI

Jun 11

INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

arXiv:2606. 11440v1 Announce Type: new Abstract: Existing multi-agent LLM orchestration methods, ranging from brute-force ensembles to learned routers, select models and topologies based on task and model features.

By Ahasan Kabir, Jiaqi Xue, Mengxin Zheng, Qian Lou

llms agents reinforcement-learning benchmarks

arXiv AI

Jul 23

Fine-grained Computation-Communication Overlap via Tile-level Signaling and Scheduling for Mixture-of-Experts

arXiv:2607. 19539v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architectures increase model capacity without proportionally increasing computation cost and have become a key building block for scaling large language models (LLMs) to trillion-parameter regimes.

By Minyu Cui, Anna Wingkvist, Morgan Ericsson

llms benchmarks

Related stories

FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs

Lodestar: An Online-Learning LLM Inference Router

Don't Let a Few Network Failures Slow the Entire AllReduce

Decentralised AI Training and Inference with BlockTrain

INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

Fine-grained Computation-Communication Overlap via Tile-level Signaling and Scheduling for Mixture-of-Experts