The paper introduces Smooth‑L1 regularized Echo State Networks (SL1ESN) that encourage sparsity in the readout layer while remaining differentiable for gradient‑based training. Using a twice‑differentiable Smooth‑L1 approximation, SL1ESN is trained centrally and via a second‑order federated aggregation scheme, achieving 75–95% sparsity across seven time‑series classification benchmarks. The resulting sparse models reduce storage, inference cost, and communication overhead, with only minor accuracy loss compared to dense baselines.
arXiv:2509. 24122v3 Announce Type: replace Abstract: At the heart of time-series forecasting (TSF) lies a fundamental challenge: how can models efficiently and effectively capture long-range temporal dependencies across ever-growing sequences?
arXiv:2606. 24969v1 Announce Type: new Abstract: While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states.
arXiv:2606. 01155v1 Announce Type: cross Abstract: Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with limited data is not.
arXiv:2508. 21172v3 Announce Type: replace-cross Abstract: Echo State Networks (ESNs) are a particular type of untrained Recurrent Neural Networks (RNNs) within the Reservoir Computing (RC) framework, popular for their fast and efficient learning.
arXiv:2608. 03324v1 Announce Type: new Abstract: Federated learning enables collaborative model training across distributed edge devices while strictly preserving data privacy.
The paper introduces S‑SALAAD, a framework that jointly learns model weights, low‑rank, and structured sparse components during training. It supports hardware‑compatible sparsity patterns and couples structure learning with budget‑aware compression and dedicated CPU/GPU kernels, allowing efficient inference from a single checkpoint across multiple memory budgets. Experiments on Llama2 and Qwen3 demonstrate improved perplexity, memory usage, and latency on various hardware platforms compared to baselines.
DART-FL is a multitask federated learning framework designed for edge devices that must balance online inference and model training under limited resources. It dynamically allocates resources between inference and training based on current inference backlog and service capacity, then distributes remaining training capacity among tasks using a queue‑aware scheduler that adjusts loss weights. Experiments on image classification datasets with synthetic and real workloads show that DART‑FL adapts to bursty inference demand, improving accuracy for high‑demand tasks while preserving overall multitask performance.
arXiv:2512. 12737v2 Announce Type: replace Abstract: Decentralized federated learning (DFL) enables collaborative model training without a central server, but converges slowly under statistical heterogeneity.
arXiv:2610.19997v1 Announce Type: new Abstract: Efficient sequence modeling remains a fundamental challenge in machine learning. Despite the success of Transformers, recurrent neural networks (RNNs)...
The paper introduces FedSWE, a federated learning algorithm designed to handle non‑stationary and heterogeneous client availability without requiring prior real‑time knowledge of which devices are online. FedSWE compensates for missed computations, stabilizes global updates, and mixes local updates through implicit gossiping, all while adding only modest memory and computational overhead. The authors prove that FedSWE converges to a stationary point for non‑convex objectives and achieves linear speedup in certain scenarios, and they validate these claims with experiments on real‑world datasets featuring diverse client unavailability patterns.
Federated zeroth-order optimization (FedZOO) reduces the memory and communication costs of collaborative training for edge intelligence by avoiding backpropagation while communicating only random seed...
arXiv:2610.35211v1 Announce Type: cross Abstract: Federated zeroth-order optimization (FedZOO) reduces the memory and communication costs of collaborative training for edge intelligence by avoiding b...