arXiv AI
Jun 2
Evaluation of Baseline Methods for IDD-based SSD External Memory Search
arXiv:2606. 01840v1 Announce Type: new Abstract: Many difficult search problems cannot be solved by algorithms such as A* using only RAM.
How to Optimize Vector Search When RAM Gets Too Expensive: On-Disk vs. In-Memory ANN Indexes
Architecting cost-effective infrastructure by navigating the latency and storage trade-offs of HNSW, SPANN, and DiskANN The post How to Optimize Vector Search When RAM Gets Too Expensive: On-Disk vs. In-Memory ANN Indexes appeared first on Towards Data Science .
Related stories
Towards Data Science
Jul 1
What Can We Do When Memory Becomes the New Bottleneck in Data Engineering?
How Pandas chunking, Dask, and Polars help process millions of records when adding more compute isn't an option. The post What Can We Do When Memory Becomes the New Bottleneck in Data Engineering?
arXiv Machine Learning
Jun 9
Aperon Technical Report: Hierarchical No-Pointer Tangent-Local Search for High-Dimensional Approximate Nearest Neighbors
arXiv:2606. 08813v1 Announce Type: cross Abstract: We present HNTL (Hierarchical No-pointer Tangent-Local), the core vector indexing and candidate generation framework of the Aperon vector memory system.
arXiv Machine Learning
Sep 23
Memoization Without Keys: Compact, Out-of-Core Tables for Functions of Sorted Arguments
arXiv:2609.20276v2 Announce Type: replace-cross Abstract: Memoizing an expensive function of a sorted score vector is a data-structure problem before it is a numerical one: at a billion gridpoints, a...
arXiv AI
Aug 25
MEMONDEMAND: A Memory Management System for Large-Scale Enterprise Data
arXiv:2608.22141v1 Announce Type: new Abstract: Enterprise repositories are large, heteroge- neous, and continuously updated, making re- trieval difficult when efficient access, source- faithful evid...
arXiv AI
Aug 18
Stop Indexing at Full Precision: Revisiting Clustering for Vector Embeddings
arXiv:2608. 14648v1 Announce Type: cross Abstract: In this study, we revisit three widely used techniques in vector search and utilize them to optimize vector embedding indexing through clustering: dimensionality reduction, quantization, and dimension pruning.
arXiv Machine Learning
Jul 14
FlashTrie: A GPU-Accelerated Constrained Beam Search for Generative Retrieval
arXiv:2607. 10044v1 Announce Type: new Abstract: Constrained decoding is essential in generative retrieval, where document identifiers generated directly from a query must exactly match a predefined library of valid IDs.
arXiv AI
Jul 21
When to Use Which? Benchmarking Optimisers for Configurable Systems under Varying Budgets
arXiv:2607. 16476v1 Announce Type: cross Abstract: Software configuration tuning is crucial for optimising system performance, and various optimisers have emerged over the last decade.
arXiv Machine Learning
Jun 11
Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elite
arXiv:2606. 11257v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) pipelines are compute-intensive, combining embedding, retrieval, reranking, and large language model (LLM) generation.
arXiv AI
Jul 17
Long-Context Fine-Tuning with Limited VRAM
arXiv:2607. 15105v1 Announce Type: new Abstract: Parameter-efficient fine-tuning reduces model and optimizer memory, but dense attention still makes long training sequences expensive.
arXiv Machine Learning
Aug 26
A Feature-Major Codebook for Memory-Efficient Sparse-Binary Self-Organizing Maps: Scaling a MEDLINE Atlas to 1.05 Million Neurons on a Single Consumer GPU
The paper presents a memory‑efficient sparse‑binary self‑organising map (SOM) that scales a MEDLINE atlas to over a million neurons on a single consumer GPU. By re‑ordering the codebook into a feature‑major layout, the authors accelerate the best‑matching‑unit search by 4.5–8.5× without increasing quantisation error, enabling training of a 1,048,576‑neuron SOM in 72 s on a 24 GB GPU. The approach outperforms existing cuSPARSE and CPU‑based SOM implementations, achieving the largest SOM reported to date and demonstrating that resolution limits are computational rather than data‑driven.
arXiv Machine Learning
Aug 18
NeuRoute: Logit-Guided Neural Routing for Billion-Scale Vector Search with Sub-Hour Index Construction
arXiv:2608. 15438v1 Announce Type: cross Abstract: Building approximate nearest neighbor (ANN) indexes at billion scale is often dominated by expensive global clustering or graph construction, making time-to-index a first-order systems concern.