← Back to all news
arXiv Machine Learning

BLOKS: Scaling Kernel Solvers with Sketched Block Preconditioning

BLOKS is a Conjugate Gradient-based solver for Nyström‑sketched Kernel Ridge Regression that introduces a novel preconditioner combining block Jacobi approximation with sketching. This preconditioner’s memory cost is independent of dataset size and model parameters, enabling scaling to large models while maintaining GPU acceleration. Experiments on standard large‑scale datasets show BLOKS achieves accuracy comparable to or better than state‑of‑the‑art kernel solvers with favorable time/memory complexity.

Related stories

arXiv Machine Learning
Oct 19

Scalable Levenberg-Marquardt Training Framework for Deep Learning

The paper introduces LeMA, a distributed Levenberg-Marquardt algorithm designed for large-scale neural network training on multi‑GPU systems. By deriving an exact dual formulation that eliminates memory dependence on parameter count and implementing a Jacobian sharding strategy, LeMA reduces the memory footprint from quadratic in parameters to quadratic in batch size. This enables exact LMA steps for billion‑parameter models, achieving a 7.97× speedup and 99.7% strong scaling efficiency on an 8‑GPU node for a convolutional network with over a billion parameters.

By Xianghao Wang, Johan Barth\'elemy, John Taylor
efficiency
More like this →
arXiv Machine Learning
Jul 21

One-shot acceleration of transient PDE solvers via online-learned preconditioners

arXiv:2509. 08765v4 Announce Type: replace-cross Abstract: Data-driven acceleration of scientific computing workflows has been a high-profile aim of machine learning (ML) for science, with numerical simulation of transient partial differential equations (PDEs) being one of the main applications.

By Mikhail Khodak, Min Ki Jung, Brian Wynne, Edmond Chow, Egemen Kolemen
reinforcement-learning
More like this →
arXiv Machine Learning
Sep 25

A Neural Hierarchical-Matrix Preconditioner for Real-Time GPU Solves

The paper presents a neural hierarchical‑matrix preconditioner designed for real‑time GPU solves of sparse symmetric positive‑definite systems that change each frame. By training a graph‑and‑attention network to predict an SPD approximate inverse in H²‑matrix format, the method achieves linear‑time inference and application, outperforming traditional multigrid setup times and local preconditioners. Experiments on 3D mesh diffusion problems show the preconditioner reduces conjugate‑gradient iterations from 116 to 33 and enables 120 fps real‑time performance for up to 3,647 unknowns.

By Carl Osborne, Minghao Guo, Crystal Owens, Wojciech Matusik
diffusion
More like this →