Compressed context is usually carried as human-readable text or as rendered images that must be decoded, even when its consumer is a language model. We introduce LatentPress, which writes conversation...
OpticalRAG is a retrieval‑augmented generation framework that compresses documents by rendering them as images and encoding them with a frozen vision encoder. It retrieves visual tokens using an Encoder‑Bridged Retrieval mechanism and condenses them into as few as 16 query‑conditioned prefix tokens via Query‑Driven Distillation, avoiding costly cross‑modal training. The method achieves state‑of‑the‑art compression‑aware accuracy on LongBench‑v2, BAMBOO, and LooGLE‑v2 while adding only 16 tokens to the LLM input.
By Senhao Liu, Yuheng Zhang, Chunyu Wei, Yueguo Chen, Xinran Zhang
The paper introduces LOHA, a context layout that compresses older tool observations into soft tokens while keeping the agent’s own turns and the last K observations in plain text, and ACD, a training method that distills full‑text predictions into this latent representation while anchoring behavior on plain text. This approach reduces context per call by up to 57% without significant loss in resolve rates, and improves instance throughput in single‑GPU serving. Experiments on SWE‑bench Verified show that K=3 yields a 43–57% compression with only modest performance impact, while larger windows favor task performance over compression.
By Zhensheng Zou (Peking University), Guoqing Wang (Peking University), Dan Hao (Peking University)
arXiv:2609.36651v1 Announce Type: cross
Abstract: Long-context reasoning in large language models incurs substantial computation and memory costs. Visual text compression (VTC) reduces input length b...
By FangZhi Zhong, Xuerui Qiu, Yuqi Pan, Ya Liu, Shaowei Gu, Bo Xu, Guoqi Li
The paper investigates the disparity between a vision‑language model’s ability to recognize text in compressed images and its ability to use that text for answering questions, a phenomenon termed the read‑use gap. Experiments show that while models can transcribe images accurately, they often fail to answer based on the transcription, and that recognition saturates at lower pixel resolutions than usage. The authors propose ALOUD, a method that allocates visual resolution to evidence detection and language tokens to reasoning, which doubles the accuracy of a frozen Qwen3‑VL‑8B on heavily compressed documents without retaining the original text.
By Zhiyun Shi
External memory effectively grounds large language models (LLMs) and vision-language models (VLMs)-based question answering (QA) in relevant multimodal evidence. However, existing memory paradigms represent each memory item in raw text and image forms, so retrieval-based systems must pass the retrieved text or images to the generation LLMs/VLMs, resulting in high token consumption and storage pressure, making it unaffordable for resource-constrained applications.