arXiv AI
Jun 2
Evaluation of Baseline Methods for IDD-based SSD External Memory Search
arXiv:2606. 01840v1 Announce Type: new Abstract: Many difficult search problems cannot be solved by algorithms such as A* using only RAM.
How to Optimize Vector Search When RAM Gets Too Expensive: On-Disk vs. In-Memory ANN Indexes
Architecting cost-effective infrastructure by navigating the latency and storage trade-offs of HNSW, SPANN, and DiskANN The post How to Optimize Vector Search When RAM Gets Too Expensive: On-Disk vs. In-Memory ANN Indexes appeared first on Towards Data Science .
Related stories
Towards Data Science
Jul 1
What Can We Do When Memory Becomes the New Bottleneck in Data Engineering?
How Pandas chunking, Dask, and Polars help process millions of records when adding more compute isn't an option. The post What Can We Do When Memory Becomes the New Bottleneck in Data Engineering?
arXiv Machine Learning
Jun 9
Aperon Technical Report: Hierarchical No-Pointer Tangent-Local Search for High-Dimensional Approximate Nearest Neighbors
arXiv:2606. 08813v1 Announce Type: cross Abstract: We present HNTL (Hierarchical No-pointer Tangent-Local), the core vector indexing and candidate generation framework of the Aperon vector memory system.
arXiv AI
1d ago
Stop Indexing at Full Precision: Revisiting Clustering for Vector Embeddings
arXiv:2608. 14648v1 Announce Type: cross Abstract: In this study, we revisit three widely used techniques in vector search and utilize them to optimize vector embedding indexing through clustering: dimensionality reduction, quantization, and dimension pruning.
arXiv Machine Learning
Jul 14
FlashTrie: A GPU-Accelerated Constrained Beam Search for Generative Retrieval
arXiv:2607. 10044v1 Announce Type: new Abstract: Constrained decoding is essential in generative retrieval, where document identifiers generated directly from a query must exactly match a predefined library of valid IDs.
arXiv AI
Jul 21
When to Use Which? Benchmarking Optimisers for Configurable Systems under Varying Budgets
arXiv:2607. 16476v1 Announce Type: cross Abstract: Software configuration tuning is crucial for optimising system performance, and various optimisers have emerged over the last decade.
arXiv Machine Learning
Jun 11
Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elite
arXiv:2606. 11257v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) pipelines are compute-intensive, combining embedding, retrieval, reranking, and large language model (LLM) generation.
arXiv AI
Jul 17
Long-Context Fine-Tuning with Limited VRAM
arXiv:2607. 15105v1 Announce Type: new Abstract: Parameter-efficient fine-tuning reduces model and optimizer memory, but dense attention still makes long training sequences expensive.
arXiv Machine Learning
1d ago
NeuRoute: Logit-Guided Neural Routing for Billion-Scale Vector Search with Sub-Hour Index Construction
arXiv:2608. 15438v1 Announce Type: cross Abstract: Building approximate nearest neighbor (ANN) indexes at billion scale is often dominated by expensive global clustering or graph construction, making time-to-index a first-order systems concern.
arXiv AI
Jun 9
SIFT: Selective-Index For Fast Compute of RAG Prefill by Exploiting Attention Invariance
arXiv:2606. 09441v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) injects LLM queries with relevant documents to improve response quality.
arXiv AI
Aug 3
GPU-Accelerated ANNS: Quantized for Speed, Built for Change
arXiv:2601. 07048v5 Announce Type: replace-cross Abstract: Approximate nearest neighbor search (ANNS) is a core problem in machine learning and information retrieval applications.
arXiv Machine Learning
Jul 15
LiteTopK: Exploiting the Curse of Dimensionality for a Fused Indexer-TopK Kernel in Long-Context Sparse Attention
arXiv:2607. 11976v1 Announce Type: new Abstract: Indexer-TopK, the operation to compute the scores and select the top-k candidates, is widely used by sparse attention kernels in large language models and vector retrieval in recommendation systems and vector databases.