arXiv:2607. 13332v1 Announce Type: new Abstract: Training large language models at the multi-billion to trillion parameter scale is confined to datacenters, where data-parallel (DP) and model-parallel (MP) techniques presume homogeneous accelerators, high-speed interconnects, and a single orchestrating entity.
arXiv:2610.33475v1 Announce Type: cross Abstract: The widespread deployment of foundational Large Language Models (LLM) across cloud datacenters marks a fundamental paradigm shift from stateless deep...
arXiv:2606. 00946v1 Announce Type: cross Abstract: Efficiently serving large language model (LLM) inference tasks is crucial both for user-perceived latency such as time-to-first-token (TTFT) and for GPU utilization.
ShardMeter is a lightweight analytical performance model that predicts end-to-end runtime for transformer-based workloads across sharded, distributed, and decentralized training setups. By taking a model’s characteristics and a target hardware topology as input, it estimates per-GPU and per-island throughput, training cost, total wall-clock time, and pinpoints performance bottlenecks. The model reveals diminishing-return regimes with increasing island size, quantifies compute- versus communication-bound scaling, evaluates hyperparameter trade-offs, and models cost-throughput for large-scale decentralized training, enabling rapid exploration of configuration space and near-optimal deployment plans.
arXiv:2608.22188v1 Announce Type: cross Abstract: Split Federated Fine-tuning (SFF) is a promising paradigm for scaling Large Language Models (LLMs) by partitioning model depth between resource-const...
arXiv:2602. 24044v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently.
arXiv:2606. 27743v1 Announce Type: cross Abstract: Large Language Models (LLMs) inference is typically deployed under a static resource assumption, where models execute a fixed computational graph regardless of the runtime environment.
InferSlice is a reference architecture that enables Service-Level Objective (SLO)-driven large language model (LLM) inference across geographically distributed telco infrastructure. It integrates network slicing, GPU partitioning, elastic serving, and observability to address sovereignty, compliance, latency, and resource‑locality constraints that limit centralized cloud deployments. The architecture was evaluated on the JOINER UK national‑scale wide‑area network, demonstrating how different parallelism strategies and resource placements affect latency‑throughput regimes and scale‑out times.
The paper proposes two hybrid algorithms, FL+FSDP and FL+HSDP, that combine sharded data parallelism with federated learning-style aggregations to reduce communication overhead in large-scale AI training. By partitioning GPUs into loosely‑coupled federation groups, the methods keep inter‑group traffic minimal while maintaining a bounded global batch size. Experiments on a Llama3.1 8B model trained on 512 A100 GPUs show up to 8.04× faster data processing and 4.48 lower evaluation perplexity compared to conventional sharded DP.
arXiv:2609.08307v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires bala...
The paper proposes two hybrid algorithms, FL+FSDP and FL+HSDP, that combine sharded data parallelism with federated learning-style aggregations to reduce communication overhead in large-scale AI training. By forming loosely‑coupled federation groups, the methods keep inter‑group traffic minimal while maintaining a bounded global batch size. Experiments on a Llama3.1 8B model trained on 512 A100 GPUs show up to 8.04× faster data processing and 4.48 lower evaluation perplexity compared to traditional sharded DP approaches.
arXiv:2411. 02908v2 Announce Type: replace Abstract: Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distributed training.