arXiv Machine Learning
LACS: Distribution-Sensitive Label-Agnostic Confidence Selection for Dynamic Data Pruning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 26
Calibration-Preserving Pruning: Compression as a Reliability Contract
arXiv:2608.23744v1 Announce Type: new Abstract: Split conformal prediction, not the pruning rule, supplies finite-sample marginal coverage once a pruned model is fixed independently of the conformal...
arXiv Machine Learning
Oct 22
Stochastic Pruning of the DPO Computation Graph: 1.4x Faster Training and Lower Peak Memory
arXiv:2610.29538v1 Announce Type: new Abstract: Direct preference optimization (DPO) incurs training costs in output scoring, sequence execution, and backpropagation, so reducing supervision alone do...
arXiv Computer Vision
Oct 16
Decision-FLIP: Budget-Free Structured Pruning for Gaussian Occupancy Models
Decision‑FLIP is a budget‑free pruning framework for Gaussian occupancy models that scores each persistent Gaussian row by how many voxel decisions change when the row is removed. It uses a guarded Otsu split on the log‑score distribution to select survivors without a target, then compiles the survivors into a smaller checkpoint, reducing bank size and latency. Across multiple model‑dataset settings, the method removes 12–79% of rows, cuts latency by 7–45%, and saves up to 8.0% of whole‑model compute while keeping semantic mIoU within 0.18 points of the original.
arXiv Machine Learning
Jul 15
Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning
arXiv:2607. 12204v1 Announce Type: new Abstract: Attention can be viewed as an online learner over context, yet existing test-time memories cannot certify that dropping a token leaves outputs unchanged or delete its influence outright.
arXiv AI
Sep 7
Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference
The paper demonstrates that layer dropout, also known as stochastic depth, can be effectively used in state‑of‑the‑art large language model (LLM) training. By optimizing the layer distribution, schedule, and optimizer settings, the authors show that layer dropout can reduce training loss while saving up to 25 % of training FLOPs. Additionally, layer dropout enables post‑training optimizations such as early exit and self‑speculative decoding, achieving up to 1.5× inference speedup with negligible accuracy loss across models ranging from 271 M to 8.2 B parameters and datasets up to 160 B tokens.
arXiv Machine Learning
1d ago
Recursive Sparse Attention: Million-Token Length Extrapolation at Logarithmic Cost
arXiv:2611. 07232v1 Announce Type: cross Abstract: At context lengths of about 1M tokens, dense attention is costly in prefill and decoding, and training typically relies on long documents, which become scarcer as the target length increases.