arXiv Machine Learning

Amortized Maximum Inner Product Search with Learned Support Functions

arXiv:2603. 08001v3 Announce Type: replace Abstract: Maximum inner product search (MIPS) is a crucial subroutine in machine learning, requiring the identification of a vector taken within a database (the keys) that best aligns with a given query.

Hugging Face Trending Papers
Aug 20

Learning Early-to-Final Solution Consistency for MILP Acceleration

Mixed-Integer Linear Programming (MILP) is a fundamental problem class in operations research and combinatorial optimization, with broad applications to industrial decision-making. Owing to their NP-hardness, however, modern solvers may struggle to find high-quality solutions for challenging MILP instances within practical time limits.

arXiv Machine Learning
Aug 27

MetaSieve: Faster Relational Deep Learning through SQL-Based Metapath Selection

MetaSieve is a metapath selection layer that reduces subgraph size in relational deep learning by pruning uninformative metapaths using SQL join and aggregation statistics. It scores candidate metapath extensions with a lightweight function that favors informative yet lightweight paths, discarding those below a threshold. The method is independent of GNN parameters and, when applied to the RelBench benchmark, consistently cuts per‑epoch training time while preserving or improving accuracy.

By Fahim Shahriar Khan, Ashraf Aboulnaga
arXiv AI
Jul 24

SOAP, Muon, and Beyond: Pushing LLM Pretraining Scales

arXiv:2607. 20548v1 Announce Type: cross Abstract: Higher-order optimizers such as Muon and SOAP offer faster convergence than AdamW, but their computational cost and numerical stability challenges have limited adoption at scale.

By Mikail Khona, Aditya Vavre, Boxiang Wang, Deyu Fu, Hao Wu, Mike Chrzanowski, Bryan Catanzaro, Dheevatsa Mudigere, Jeff Pool, Michael Lightstone, Mohammad Shoeybi, Mostofa Patwary, Nima Tajbakhsh, Tijmen Blankevoort