The paper reports a Size‑Fidelity Paradox in large‑language‑model (LLM) context compression: larger compressors can produce lower reconstruction error yet yield less faithful reconstructed contexts. Two key mechanisms drive this effect: knowledge overwriting, where larger models replace source facts with their own beliefs, and semantic drift, where they paraphrase or restructure content. Across 27 compressor setups, mid‑sized models often recover contexts more faithfully, and analysis of embedding geometry shows that larger models organize memory into broader semantic subspaces, leading to ambiguous representations that weaken recovery.
arXiv:2608. 12218v1 Announce Type: cross Abstract: Large language models are increasingly trained and deployed with long contexts that span documents, code repositories, and interaction histories.
arXiv:2603. 11784v2 Announce Type: replace Abstract: As scaling laws push the training of frontier large language models (LLMs) toward ever-growing data requirements, training pipelines are approaching a regime where much of the publicly available online text may be consumed.
The paper examines whether context compression, which encodes inputs into compact representations while preserving semantic content, maintains refusal alignment in instruction-tuned language models. It finds that encoder-based compression consistently weakens refusal behavior across various model families and architectures, even though reconstruction fidelity remains high. Mechanistic analysis attributes this to attenuation of activation along the decoder’s learned refusal direction, and introduces Memory Steering—a lightweight inference-time intervention that restores refusal rates without retraining.
The paper argues that large language models cannot achieve perfect reliability for any task, even with unlimited scale. It establishes that each generative task has an inherent reliability ceiling set by how much output uncertainty can be resolved from observable context, with a resolvable part that can be improved by more context and a subjective part tied to task ambiguity. The authors derive a scaling law showing that performance is limited by the scarcer resource—either training data or model capacity—and explain how this law explains phenomena such as retrieval augmentation and catastrophic forgetting.
arXiv:2609.24238v1 Announce Type: new Abstract: We reproduce and stress-test the work of Yu et al. (2023), who characterize how language models (LMs) arbitrate between memorized knowledge and contrad...
arXiv:2609.13154v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have made prompts increasingly large and complex. Techniques such as chain-of-thought reasoning (Wei et...
arXiv:2607. 14112v1 Announce Type: cross Abstract: Large language models (LLMs) are evaluated as though perfect reliability is achievable for any task given sufficient scale.
The paper investigates how large language models acquire knowledge during pre‑training, proposing that auxiliary views—reformulations of knowledge—are causally beneficial. Experiments show that repetition is essential, paraphrasing helps only at smaller batch sizes, and reallocating tokens from repetition to auxiliary views improves learning even for factual recall. The study also finds that the benefit of auxiliary views does not depend on the teacher model’s strength, identifies specific knowledge types that aid learning, and explores mechanistic effects via layer‑wise biases and compression.
arXiv:2606. 24083v1 Announce Type: cross Abstract: "Talk short.
The paper investigates how semantic meaning is arranged locally within large language models. By using two controlled probes—one cross‑lingual and one refinement probe—on four decoder‑only LLMs, it finds that after removing dominant variance directions, sentence pairs rank by semantic relatedness, semantic factors cluster around an invariant core with structured variation, refinement tightens neighborhoods, and sentence‑level versus token‑local readouts reveal complementary geometric patterns. These results suggest that LLM semantics form local anisotropic neighborhoods that contract when meaning is refined.
arXiv:2506.17871v4 Announce Type: replace-cross Abstract: Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this cons...
Context compression reduces inference cost by encoding inputs into compact representations while preserving semantic content. An open question is whether semantic preservation alone is sufficient to m...