The Flow has not summarised this story yet — read it at arXiv Computation and Language.
arXiv:2610.10304v1 Announce Type: new Abstract: We study whether latent sequence compression of prompt prefixes preserves the capabilities that large language models rely on during inference. We intr...
arXiv:2607. 08284v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them.
Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them. However, existing long-context evaluations - from Needle-in-a-Haystack (NIAH) tests to more recent multi-hop reasoning and summarization tasks - predominantly measure average-case performance, and many are either saturated or lack robustness.
The paper introduces OnQ, a low‑bit recovery pipeline for quantized large language models that uses on‑policy distillation and candidate‑wise Dynamic KL to address trajectory divergence and distributional misalignment. OnQ starts with block‑wise calibration, then lets the quantized student generate responses and receive full‑precision teacher supervision on its own prefixes, dynamically weighting forward and reverse KL losses per candidate. Experiments on Qwen3 models show that OnQ outperforms baselines on nine benchmarks, especially long‑horizon reasoning, and produces shorter, less truncated responses.
The paper introduces zip2zip++, a method for inference‑time token compression that reduces token counts using a dynamic Lempel‑Ziv‑Welch (LZW) codec while preserving language model quality. It identifies four problems with prior LZW‑based compression—embedding conflict, improper RoPE application, harmful digit merges, and segmentation ambiguity—and proposes solutions: separate input/output embeddings, size‑aware RoPE, and a selective‑merge LZW codec. Experiments on four models show that these changes halve the quality gap in perplexity and reasoning accuracy compared to the uncompressed baseline, achieving similar compression ratios.
LADDER is a new framework that combines diffusion language modeling with Graph Retrieval-Augmented Generation (GraphRAG) to enable efficient multi‑hop reasoning. It introduces an event‑driven self‑clocking retrieval mechanism that triggers graph queries only when new entities appear, and an incomplete‑query graph propagation module that aggregates multi‑hop evidence during parallel decoding. Experiments on three multi‑hop QA benchmarks show that LADDER improves exact match from 39.6% to 45.2% while reducing latency by 4.1×.