arXiv AI
Enhancing Local and Global Structure in Compressed AI Agent Memory
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Sep 16
Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families
arXiv:2609. 16391v1 Announce Type: cross Abstract: Weight-only post-training quantization is the cheapest way to shrink a retrieval embedder, and the received advice for applying it -- protect the embedding table, allocate bits by module sensitivity, prefer a ranking-aware objective over weight reconstruction -- was carried into LLM quantization largely intact.
arXiv Machine Learning
Jul 9
Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference
arXiv:2607. 07144v1 Announce Type: new Abstract: The key-value (KV) cache dominates the memory cost of long-context autoregressive inference, and a growing body of work compresses it through quantization, eviction, or offloading.
arXiv Machine Learning
Oct 28
BellowsKV: One Latent for Many Layers, Expanded on Demand
BellowsKV introduces a training‑free approach that maps the hidden states of multiple adjacent layers in a large language model to a single shared latent representation per token, allowing each layer to retrieve its own keys and values via a layer‑specific projection. The method retains the latent at full dimension and offers two compression strategies: BellowsKV‑Quant, which stores the latent at low precision, and BellowsKV‑Evict, which makes a shared eviction decision per block. Experiments on Llama‑3.1‑8B, Mistral‑7B, and Qwen2.5‑7B show that BellowsKV‑Quant preserves about 99% of full‑cache accuracy at 3.6× compression and 97‑98% at 13‑14×, outperforming existing cross‑layer methods that drop to at most 84% accuracy at 16× compression.
arXiv AI
2d ago
FlyHash for LLM Agent Memory: An Empirical Validation and a New Application
The paper evaluates FlyHash, a locality‑sensitive hashing scheme inspired by the fruit fly’s olfactory circuit, for use in large language model (LLM) agent memory retrieval. It demonstrates that FlyHash maintains a quality advantage over SimHash when bits are matched, and that this advantage persists with real agent‑memory relevance labels. However, FlyHash’s larger, slower codes and the dominance of dense‑vector indexing (HNSW) limit its practical benefits in terms of memory and latency savings.
arXiv AI
Jun 9
Projection and Quantisation: A Unifying View of Learning to Hash, from Random Projections to the RAG Era
arXiv:2510. 04127v2 Announce Type: replace-cross Abstract: Approximate nearest neighbour (ANN) search underpins large-scale retrieval, increasingly within the retrieval-augmented generation pipelines that ground large language models, yet the methods that address it have multiplied across communities until they are seldom read as a single field.
arXiv Machine Learning
Jul 7
Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models
arXiv:2607. 02893v1 Announce Type: new Abstract: Low-bit quantization shrinks language models but treats precision as a single global hyper-parameter: every weight uses the same bit-width.