The paper introduces a set of software packages in R, Python, Julia, and C++ that solve the Sorted L-One Penalized Estimation (SLOPE) problem efficiently. The packages employ a hybrid coordinate descent algorithm capable of fitting generalized linear models with various loss functions such as Gaussian, binomial, Poisson, and multinomial logistic regression. They support dense, sparse, and out‑of‑memory data structures, can compute the full SLOPE path, perform cross‑validation (including relaxed SLOPE), and are shown to outperform existing SLOPE implementations in speed on both real and simulated data.
arXiv:2609. 08136v1 Announce Type: new Abstract: This paper introduces rlaopt, a PyTorch-based package for large-scale optimization and scientific computing using randomized numerical linear algebra (RandNLA).
arXiv:2606. 06742v1 Announce Type: new Abstract: TorchKM is an open-source library for kernel machines, including support vector machines, kernel logistic regression, and kernel quantile regression, with GPU acceleration.
arXiv:2306. 14851v5 Announce Type: replace-cross Abstract: Given a high-dimensional covariate matrix and a response vector, ridge-regularized sparse linear regression selects a subset of features that explains the relationship between covariates and the response in an interpretable manner.
arXiv:2606. 08638v1 Announce Type: cross Abstract: Recent research has developed practical, parallelizable first-order methods for large scale linear programming, but performance is highly dependent on hyperparameter selection.
arXiv:2606. 27895v1 Announce Type: cross Abstract: Differentiable partial differential equation (PDE) solvers underpin solver-in-the-loop ML training, gradient-based optimal control, and inverse problems, yet the practical cost of obtaining correct, usable gradients from a given solver on a given problem is largely undocumented.
arXiv:2209. 03282v5 Announce Type: replace-cross Abstract: Accelerating the convergence of second-order optimization, particularly Newton-type methods, remains a pivotal challenge in algorithmic research.
arXiv:2608. 06912v1 Announce Type: new Abstract: The top-$k$ operation is a fundamental building block of modern sparse computation, enabling token routing, expert activation, memory selection, and attention pruning.
GRADSOLVE is an open‑source JAX library that provides fast, exact reverse‑mode gradients for low‑dimensional ordinary differential equation (ODE) ensembles on NVIDIA GPUs. It records the accepted steps of an adaptive solver and differentiates a fixed‑step replay, yielding the exact discrete adjoint at a lower computational cost than traditional checkpointed methods. Benchmarks show that GRADSOLVE’s forward kernel is 2.8× faster than DiffEqGPU.jl, and its gradient computation is 5.6–14.1× faster than Diffrax’s checkpointed adjoint while maintaining matched forward‑state accuracy across multiple GPU generations.
arXiv:2606. 30328v1 Announce Type: cross Abstract: Rapid prototyping of algorithms is a critical step in modern machine learning.
arXiv:2601. 22813v2 Announce Type: replace Abstract: The NVFP4 lower-precision format, supported in hardware by NVIDIA Blackwell GPUs, promises to allow, for the first time, end-to-end fully-quantized pre-training of massive models such as LLMs.
arXiv:2606. 19878v1 Announce Type: new Abstract: Recent work on first-order optimizers for empirical risk minimization (ERM) has suggested that smoothness of ERM loss functions in the training data, rather than in the optimization parameters, can be leveraged to improve the oracle complexity of gradient descent (GD) methods.
The paper introduces a GPU-resident, batched Levenberg–Marquardt solver that efficiently optimizes constants in tree-based genetic programming for symbolic regression. By using reverse-mode automatic differentiation to assemble per-tree Jacobians in a single backward sweep, the solver’s per-iteration cost becomes independent of the number of constants per tree, achieving up to 510,000 trees per second on an NVIDIA A100. Integrated into EvoGP, the solver enables end-to-end search that recovers governing equations on 10 of 18 constructed problems, a significant improvement over stock EvoGP.