arXiv Machine Learning
Jul 14
Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Data
arXiv:2607. 11883v1 Announce Type: new Abstract: Compression is fundamental to intelligence.
Dynamics as Code: On Model Compression via Dynamic System
arXiv:2610. 11115v1 Announce Type: new Abstract: The escalating size of pretrained neural networks has rendered model compression a prerequisite for deployment under stringent memory and compute constraints.
Related stories
arXiv AI
Jun 2
Neural Network Compression by Approximate Differential Equivalence
arXiv:2606. 01402v1 Announce Type: cross Abstract: Neural network compression is commonly achieved by pruning parameters based on local importance scores, e.
Hugging Face Trending Papers
Aug 25
Compression Trinity: Exploring Sparsity, Quantization, and Low-Rank Approximations for LLM Compression
Prohibitive computational and environmental costs impede the scalable deployment of Large Language Models (LLMs). Traditional compression techniques (sparsity, quantization, low-rank approximations) a...
arXiv Machine Learning
Aug 27
DAW: Dynamics-Aware Weighting for Deep Learning Forecasts of Chaotic Systems
arXiv:2608. 22277v2 Announce Type: replace Abstract: Deep learning surrogates for forecasting chaotic dynamical systems suffer from catastrophic error accumulation over long-term autoregressive rollouts.
arXiv AI
Aug 26
Compression Trinity: Exploring Sparsity, Quantization, and Low-Rank Approximations for LLM Compression
The paper introduces the "Compression Trinity," a unified framework that jointly applies sparsity, quantization, and low‑rank approximations to compress large language models. It presents several methods—MKOR, SLoPe, OPTIMA, PATCH, and SLiM—that leverage these three pillars to accelerate training, reduce memory bandwidth, and recover accuracy, achieving significant speedups and accuracy gains over existing techniques. The results demonstrate that combining all three compression strategies is essential for efficient, scalable, high‑performance LLM deployment.
arXiv Machine Learning
Sep 30
Dynamics-Aware Weighting for Deep Learning Forecasts of Chaotic Systems
arXiv:2608. 22277v3 Announce Type: replace Abstract: Deep learning surrogates have become powerful tools for simulating and forecasting complex dynamical systems, yet their utility remains limited by catastrophic error accumulation during long-term autoregressive rollouts.
arXiv Machine Learning
Aug 4
SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits
arXiv:2608. 00859v1 Announce Type: new Abstract: Kolmogorov--Arnold Networks (KANs) replace scalar edge weights with learnable univariate functions parameterized by multiple basis coefficients.
Hugging Face Trending Papers
Jul 13
Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Data
Compression is fundamental to intelligence. A model that can represent its training data as a short code has discovered regularities that enable generalization.
arXiv Machine Learning
Jul 17
Gibbs randomness-compression proposition
arXiv:2505. 23869v4 Announce Type: replace-cross Abstract: A proposition that connects randomness and compression is put forward via Gibbs entropy over set of measurement vectors associated with a compression process.
arXiv AI
Jul 8
Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests
arXiv:2607. 05457v1 Announce Type: cross Abstract: Deep neural networks often contain substantial hidden-state redundancy, but most compression methods operate directly on weights, neurons, or quantised representations without explicitly characterising the dynamical role of internal states.
arXiv Machine Learning
Sep 25
Beyond Compression: Training Latent Representations for Stable Long-Horizon Rollout in Neural Surrogate Solvers
The paper investigates why latent neural surrogate solvers, which compress physical system dynamics into a lower‑dimensional space, often fail during long‑horizon autoregressive rollouts. It demonstrates that training the latent representation only for reconstruction leads to instability, and proposes a set of training interventions—Koopman operator learning, Hamming noise injection, and multi‑step rollout fine‑tuning—that align the latent space with long‑horizon forecasting. These interventions reduce long‑rollout error by about 40 % and achieve accuracy comparable to full‑resolution models while using far fewer floating‑point operations and GPU memory, enabling stable extrapolation in mesoscale crystal‑plasticity simulations of high‑cycle fatigue.
arXiv Machine Learning
Sep 11
LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry
LILA (Latent-Informed Layer Analysis) introduces a calibration‑free method for structured pruning of large language models by scoring neuron importance using the Kolmogorov–Smirnov distance between singular value distributions of full and neuron‑ablated feed‑forward network weight matrices. The approach requires no training, calibration data, or auxiliary networks, and outperforms existing methods such as PruneNet and SliceGPT on LLaMA‑2‑7B and Phi‑2 at various sparsity levels. After a single epoch of LoRA fine‑tuning, LILA matches heavily calibrated baselines, and a Neural Tangent Kernel analysis provides theoretical support for its spectral importance criterion. Additionally, LILA can dynamically allocate sparsity budgets, achieving state‑of‑the‑art generative preservation and revealing architectural bottlenecks at higher compression.