arXiv Machine Learning June 5, 2026 By Paul Janson, Edouard Oyallon, Eugene Belilovsky

Learned Subspace Compression for Communication-Efficient Pipeline Parallelism

Read the original on arXiv Machine Learning →

arXiv:2606. 05484v1 Announce Type: new Abstract: Pipeline parallelism enables training of large language models that exceed single-device memory, yet inter-stage activation communication becomes the dominant bottleneck when trained on low-bandwidth networks.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 16

Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training

arXiv:2606. 16384v1 Announce Type: new Abstract: Pretraining language models with extended context windows enhances their ability to leverage rich information during generation.

By Sameera Ramasinghe, Ajanthan Thalaiyasingam, Hadi Mohaghegh Dolatabadi, Gil Avraham, Violetta Shevchenko, Yan Zuo, Chamin Hewa Koneputugodage, Alexander Long

llms

arXiv Machine Learning

Jun 2

ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression

arXiv:2606. 00494v1 Announce Type: new Abstract: Post-Training Quantization (PTQ) and Low-Rank Adaptation (LoRA) constitute the standard pipeline for efficient Large Language Model (LLM) deployment.

By Wneya Yu, Chao Zhang, Li Wang, Samson Lasaulce, Merouane Debbah

llms fine-tuning efficiency

arXiv Machine Learning

Jul 10

Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelism

arXiv:2506. 01260v3 Announce Type: replace Abstract: Scaling models has led to significant advancements in deep learning, but training these models in decentralized settings remains challenging due to communication bottlenecks.

By Sameera Ramasinghe, Thalaiyasingam Ajanthan, Gil Avraham, Yan Zuo, Alexander Long

llms

arXiv Machine Learning

Jun 2

LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models

arXiv:2606. 00573v1 Announce Type: new Abstract: Vision-language models (VLMs) deliver strong multimodal reasoning capabilities, but their large computational cost and high parameter counts make deployment challenging on resource-constrained devices.

By Haiyu Wang, Yutong Wang, Leshu Li, Yihui Ren, Sai Qian Zhang

llms efficiency multimodal

arXiv Machine Learning

Jul 21

AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning

arXiv:2607. 17913v1 Announce Type: cross Abstract: Distributed Fine-Tuning (DFT) of large-scale Foundation Models (FMs) on resource-constrained edge devices is limited by local compute constraints and communication overhead.

By Bas Meuwissen, Vasileios Tsouvalas, Nirvana Meratnia

fine-tuning efficiency safety

arXiv Machine Learning

Jul 9

Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism

arXiv:2506. 01260v2 Announce Type: replace Abstract: Scaling models has led to significant advancements in deep learning, but training these models in decentralized settings remains challenging due to communication bottlenecks.

By Sameera Ramasinghe, Thalaiyasingam Ajanthan, Gil Avraham, Yan Zuo, Alexander Long

llms

Related stories

Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training

ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression

Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelism

LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models

AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning

Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism