Hugging Face Trending Papers

LatentPress: Context Compression Beyond Text and Vision

arXiv AI
Sep 2

LatentPress: Context Compression Beyond Text and Vision

LatentPress compresses conversational histories and long documents into continuous memory tokens that a frozen decoder can read directly, eliminating the need for text reconstruction at inference. The method achieves 4–16× compression with only a small adapter (0.1% of the decoder’s parameters) and outperforms text summaries and OCR-based compression on LongMemEval and LongBench-QA benchmarks. Writing and reading are significantly faster than traditional text summarization or OCR reconstruction, demonstrating a practical machine-facing context interface beyond text and vision.

By Zhengze Zhou, Hejian Sang
arXiv Computation and Language
6d ago

OpticalRAG: Pixel-Space Compression for Token-Efficient Retrieval-Augmented Generation

OpticalRAG is a retrieval‑augmented generation framework that compresses documents by rendering them as images and encoding them with a frozen vision encoder. It retrieves visual tokens using an Encoder‑Bridged Retrieval mechanism and condenses them into as few as 16 query‑conditioned prefix tokens via Query‑Driven Distillation, avoiding costly cross‑modal training. The method achieves state‑of‑the‑art compression‑aware accuracy on LongBench‑v2, BAMBOO, and LooGLE‑v2 while adding only 16 tokens to the LLM input.

By Senhao Liu, Yuheng Zhang, Chunyu Wei, Yueguo Chen, Xinran Zhang
arXiv AI
Sep 28

Compress What You See, Not What You Say: Anchored Context Distillation for Latent-Observation Software Engineering Agents

The paper introduces LOHA, a context layout that compresses older tool observations into soft tokens while keeping the agent’s own turns and the last K observations in plain text, and ACD, a training method that distills full‑text predictions into this latent representation while anchoring behavior on plain text. This approach reduces context per call by up to 57% without significant loss in resolve rates, and improves instance throughput in single‑GPU serving. Experiments on SWE‑bench Verified show that K=3 yields a 43–57% compression with only modest performance impact, while larger windows favor task performance over compression.

By Zhensheng Zou (Peking University), Guoqing Wang (Peking University), Dan Hao (Peking University)
Hugging Face Trending Papers
Jun 9

One Token per Multimodal Evidence: Latent Memory for Resource-Constrained QA

External memory effectively grounds large language models (LLMs) and vision-language models (VLMs)-based question answering (QA) in relevant multimodal evidence. However, existing memory paradigms represent each memory item in raw text and image forms, so retrieval-based systems must pass the retrieved text or images to the generation LLMs/VLMs, resulting in high token consumption and storage pressure, making it unaffordable for resource-constrained applications.

arXiv Computation and Language
Nov 4

VLMs Read Aloud: The Gap Between Recognizing and Using Visual Text

The paper investigates the disparity between a vision‑language model’s ability to recognize text in compressed images and its ability to use that text for answering questions, a phenomenon termed the read‑use gap. Experiments show that while models can transcribe images accurately, they often fail to answer based on the transcription, and that recognition saturates at lower pixel resolutions than usage. The authors propose ALOUD, a method that allocates visual resolution to evidence detection and language tokens to reasoning, which doubles the accuracy of a frozen Qwen3‑VL‑8B on heavily compressed documents without retaining the original text.

By Zhiyun Shi
arXiv Computation and Language
Sep 22

BudgetMem: Training-Free Selective Memory for Cost-Efficient Long-Context Processing in Language Models

arXiv:2511. 04919v3 Announce Type: replace Abstract: Processing long documents with large language models (LLMs) is expensive: a single query over a 100K-token document can cost from tens of cents to over a dollar in API fees, depending on the model, and memory grows linearly with context length.

By Chandra Vamsi Krishna Alla, Harish Naidu Gaddam, Manohar Kommi, Sheikh Nazib Ahmed
arXiv Machine Learning
Nov 2

Latent-Lens: Visual Perception in Small Language Models Through Latent Communication

Latent‑Lens is a lightweight codec that enables a frozen vision‑language model (VLM) to communicate visual information to a frozen small language model (SLM) directly in latent space, avoiding the lossy bottleneck of discrete text. The codec uses a domain encoder to project VLM hidden states into the SLM’s embedding space and an attention decoder (a LoRA) that teaches the SLM to attend to these visual prefixes, training only about 1.6 M parameters (0.2 % of the combined model size). On visual question answering tasks (CLEVR, GQA) and image captioning (Flickr8k), Latent‑Lens significantly outperforms text‑based communication with the same LoRA budget, achieving up to +69.8 pp improvement on VQA and +18.8 pp on captioning.

By Mo Shahdloo, Faegheh Sardari, Mathew Salvaris