arXiv AI
Aug 11
Training Variable Long Sequences with Data-Centric Parallel
arXiv:2608. 07524v1 Announce Type: new Abstract: Training deep learning models on variable long sequences poses significant computational challenges.
scDataset: Scalable Data Loading for Deep Learning on Large-Scale Single-Cell Omics
arXiv:2506. 01883v3 Announce Type: replace-cross Abstract: Training deep learning models on single-cell datasets with hundreds of millions of cells requires loading data from disk, as these datasets exceed available memory.
Related stories
arXiv Machine Learning
Jun 30
Atompack: A Storage and Distribution Layer for Read-Heavy Atomistic ML Training Datasets
arXiv:2606. 29975v1 Announce Type: new Abstract: Atomistic machine learning datasets are increasingly used for training: large immutable snapshots are read repeatedly, shuffled across epochs, staged across clusters' storage systems, and republished as reusable scientific artifacts.
Hugging Face Trending Papers
Jun 29
Atompack: A Storage and Distribution Layer for Read-Heavy Atomistic ML Training Datasets
Atomistic machine learning datasets are increasingly used for training: large immutable snapshots are read repeatedly, shuffled across epochs, staged across clusters' storage systems, and republished as reusable scientific artifacts. This workload differs from interactive scientific curation, where mutable records and ad hoc inspection are often more important than random indexed throughput.
arXiv AI
22h ago
Learnware for CSI Feedback: Scene-specific Small Models Can Do Big
arXiv:2608. 17760v1 Announce Type: cross Abstract: Intelligent channel state information (CSI) feedback is essential for realizing the high capacity and spectral efficiency goals of future 6G systems, yet existing deep learning solutions face a trade-off between model generalization and scenario-specific performance.
arXiv Machine Learning
Jul 21
EVOLVE: Efficient Learned Volume Compression with Variable-Rate Encoding on a Cross-Domain Database
arXiv:2607. 18187v1 Announce Type: cross Abstract: Large-scale scientific simulations generate volumetric data at rates that far outpace advances in storage and network bandwidth, making effective lossy compression increasingly critical.
arXiv AI
Jun 12
Versioned Late Materialization for Ultra-Long Sequence Training in Recommendation Systems at Scale
arXiv:2604. 24806v2 Announce Type: replace-cross Abstract: Modern Deep Learning Recommendation Models (DLRMs) follow scaling laws with sequence length, driving the frontier toward ultra-long User Interaction History (UIH).
arXiv Machine Learning
Jul 20
Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training
arXiv:2607. 15745v1 Announce Type: new Abstract: Common practice when training Convolutional Neural Networks (CNNs) is to use randomly shuffled mini-batches.
arXiv AI
Jun 2
When Data Is Scarce: Scaling Sparse Language Models with Repeated Training
arXiv:2606. 01155v1 Announce Type: cross Abstract: Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with limited data is not.
arXiv AI
Jun 9
FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
arXiv:2606. 09079v1 Announce Type: cross Abstract: Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving.
arXiv Machine Learning
Jun 8
BigMac: Breaking the Pareto Frontier of Compute and Memory in Multimodal LLM Training
arXiv:2605. 25451v2 Announce Type: replace Abstract: Training multimodal large language models (MLLMs) is challenged by both model and data heterogeneity.
arXiv AI
Jun 29
DataStates-LLM: Scalable Checkpointing for Transformer Models Using Composable State Providers
arXiv:2601. 16956v1 Announce Type: cross Abstract: The rapid growth of Large Transformer-based models, specifically Large Language Models (LLMs), now scaling to trillions of parameters, has necessitated training across thousands of GPUs using complex hybrid parallelism strategies (e.
arXiv AI
Jun 15
Gefen: Optimized Stochastic Optimizer
arXiv:2606. 13894v1 Announce Type: cross Abstract: AdamW is a default optimizer for modern deep learning, but its first and second moment states add roughly two parameter-sized buffers to training memory.