arXiv Machine Learning
Jun 8
Towards Tight Bounds for Streaming Attention
arXiv:2606. 07205v1 Announce Type: cross Abstract: The attention mechanism is a cornerstone of modern transformer architectures.
Sublinear Sketches for Approximate Nearest Neighbor and Kernel Density Estimation
Related stories
arXiv Machine Learning
Aug 24
Query Efficient Structured Matrix Learning
arXiv:2507.19290v2 Announce Type: replace-cross Abstract: We study the problem of learning a structured approximation (low-rank, sparse, banded, etc.) to an unknown matrix $A$ given access to matrix-...
arXiv Machine Learning
Jun 19
Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection
arXiv:2606. 19411v1 Announce Type: new Abstract: Selecting a small, diverse, high-quality subset from a massive pool of candidates is a recurring primitive in modern machine learning -- data curation and coreset selection for training and fine-tuning large models, active-learning batch acquisition, prompt and exemplar selection for in-context learning, retrieval diversification, and experimental design.
arXiv Machine Learning
Aug 14
Prof-K: Probabilistic One-Pass Filtering for Efficient Top-k Selection
arXiv:2608. 12573v1 Announce Type: new Abstract: Top-k selection is a fundamental computational primitive with applications spanning databases, information retrieval, signal processing, and modern machine learning workloads, including sparse activations and attention pruning.
arXiv Machine Learning
Jul 2
Amortized Maximum Inner Product Search with Learned Support Functions
arXiv:2603. 08001v3 Announce Type: replace Abstract: Maximum inner product search (MIPS) is a crucial subroutine in machine learning, requiring the identification of a vector taken within a database (the keys) that best aligns with a given query.
arXiv Machine Learning
Jul 14
Learning Partition Trees for Nearest Neighbor Search
arXiv:2607. 09909v1 Announce Type: cross Abstract: We study nearest neighbor search from the perspective of data-driven algorithm design: given a dataset $P \subset \mathbb{R}^d$ of size $n$ and sample access to a query distribution over $\mathbb{R}^d$, the goal is to learn a data structure optimized for queries drawn from that specific distribution.
arXiv Machine Learning
Jun 17
Amortizing Maximum Inner Product Search with Learned Support Functions
arXiv:2603. 08001v2 Announce Type: replace Abstract: Maximum inner product search (MIPS) is a crucial subroutine in machine learning, requiring the identification of a vector taken within a database (the keys) that best aligns with a given query.
arXiv Machine Learning
Jun 30
Actively Learning Halfspaces without Synthetic Data
arXiv:2509. 20848v2 Announce Type: replace-cross Abstract: In the classic point location problem, one is given an arbitrary dataset $X \subset \mathbb{R}^d$ of $n$ points with query access to an unknown halfspace $f : \mathbb{R}^d \to \{0,1\}$, and the goal is to learn the label of every point in $X$.
arXiv AI
Jul 7
Panorama: Fast-Track Nearest Neighbors
arXiv:2510. 00566v4 Announce Type: replace-cross Abstract: Approximate Nearest-Neighbor Search (ANNS) pipelines for high-dimensional neural embeddings spend the bulk of their query time in candidate verification, making it the primary bottleneck in the search process.
arXiv AI
Sep 3
A Power Law in Logarithm's Clothing: On the Scalability of Graph-Based Vector Search
arXiv:2609. 02143v1 Announce Type: cross Abstract: Most vector databases rely on graph-based indexes, notably HNSW and Vamana, for approximate nearest neighbor search.
arXiv AI
Jun 4
ANN Search: Recall What Matters
arXiv:2606. 04522v1 Announce Type: cross Abstract: Approximate nearest neighbor (ANN) search has become a core primitive in information retrieval and modern machine learning tasks, from classification to retrieval-augmented generation.
arXiv AI
Sep 10
Revisiting Thinning Methods for Kernel Learning Problems
The paper introduces Backward Kernel Herding, an algorithm that iteratively removes data points to create representative subsets for kernel learning, achieving performance comparable to state‑of‑the‑art methods while speeding up subsampling when the reduced size is less than half the original dataset. It also proposes Flexible Kernel Thinning, an extension that allows construction of subsets of any size, not just successive halvings, and demonstrates that this method often yields the best predictive performance. Experiments on Gaussian Processes and Kernel Support Vector Machines show that Backward Kernel Herding excels in training‑time efficiency, while Flexible Kernel Thinning offers superior predictive accuracy and competitive memory usage, emphasizing the need to choose a reduction strategy based on the desired trade‑off between performance, cost, and memory.