arXiv AI
SplitLite: Low-Rank Residual Compression for Split Learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 2
FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projection
FedLore introduces a communication- and memory-efficient federated learning framework that shares a low-rank optimization basis across clients each round, mitigating subspace fragmentation and enabling exact low-rank aggregation. By refreshing this shared basis across rounds, FedLore allows model updates to exceed the per-round rank budget while maintaining a provable $O(T^{-1/2})$ stationarity bound under standard assumptions. Experiments on vision and language tasks, including federated pre‑training, demonstrate that FedLore outperforms low‑rank adapter baselines and matches or surpasses full‑parameter training while reducing communication and optimizer‑state memory.
arXiv AI
Jul 21
Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
arXiv:2601. 16991v3 Announce Type: replace-cross Abstract: Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in resource-constrained environments.
arXiv Machine Learning
Jul 7
PLoRA: Efficient Concurrent LoRA Training for Large Language Models
arXiv:2508. 02932v2 Announce Type: replace Abstract: Low-Rank Adaptation (LoRA) has gained popularity as a fine-tuning approach for Large Language Models (LLMs) due to its low resource requirements and good performance.
arXiv Machine Learning
Jul 21
AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning
arXiv:2607. 17913v1 Announce Type: cross Abstract: Distributed Fine-Tuning (DFT) of large-scale Foundation Models (FMs) on resource-constrained edge devices is limited by local compute constraints and communication overhead.
arXiv Machine Learning
Oct 2
Importance-Aware Feature Sparsification for Wireless Split Learning
The paper introduces Importance-Aware Class-Balanced Sparsification (ICS) for wireless split learning, a lightweight method that ranks feature channels on the server using Grad‑CAM scores from the true‑class logit. By aggregating per‑class scores into a class‑balanced importance vector, clients can retain the top‑N feature channels without extra forward or backward passes, reducing communication overhead while mitigating head‑class bias under label skew. The authors provide a non‑asymptotic convergence bound, analyze overhead against baselines, and extend the approach to parallel split learning and transformer models, demonstrating consistent performance gains, especially under severe non‑i.i.d. data partitions.
arXiv AI
Jul 15
CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA
arXiv:2607. 11940v1 Announce Type: cross Abstract: As the scale of large pre-trained models continues to grow, fine-tuning them under limited memory budgets has become increasingly challenging.