Residual Vector-based Reconstruction as Long-Context Recall Regardless of Context Window Size

The paper introduces a long‑context recall technique that keeps GPU memory usage nearly constant regardless of context length, without requiring additional training. It reconstructs facts by leveraging residual vectors stored in the LLM’s feed‑forward layers, enabling deterministic retrieval of query‑relevant information without accessing the original document. Experiments demonstrate the method can answer single‑fact questions in two‑million‑token stories, outperforming prior approaches.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• fine-tuning