CPU Optimized Embeddings with 🤗 Optimum Intel and fastRAG

MTEB: Massive Text Embedding Benchmark

Stop Indexing at Full Precision: Revisiting Clustering for Vector Embeddings

arXiv:2608. 14648v1 Announce Type: cross Abstract: In this study, we revisit three widely used techniques in vector search and utilize them to optimize vector embedding indexing through clustering: dimensionality reduction, quantization, and dimension pruning.

HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compression

arXiv:2606. 28831v1 Announce Type: cross Abstract: Long-context LLM inference faces a fundamental conflict: head-adaptive compression algorithms (e.

MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs

arXiv:2512. 22219v2 Announce Type: replace-cross Abstract: We introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance mega-kernel.

MIVE: A Minimalist Integer Vector Engine for Softmax LayerNorm and RMSNorm Acceleration

arXiv:2606. 17781v1 Announce Type: cross Abstract: The rapid growth of Large Language Models (LLMs) has intensified the need for specialized hardware accelerators that can satisfy stringent inference latency and power constraints.

New and improved embedding model

We are excited to announce a new embedding model which is significantly more capable, cost effective, and simpler to use.