EntroPrefill: Renyi-Guided Context Pruning with Conditional Stability Guarantees for Retrieval-Augmented Generation
Read the original on arXiv Machine Learning →EntroPrefill introduces a Renyi‑guided context pruning method for retrieval‑augmented generation, coupling a proposal mechanism with constraints on discarded attention mass. The approach includes a sink‑isolated, regularized head pooling strategy that balances specialization and worst‑head coverage, and derives a computable upper bound on token removal along with finite‑sample observer guarantees for adaptive pruning layers. The paper also establishes a conditional transformer perturbation bound with Lipschitz constants, a first‑token decision‑margin corollary, and a counterexample illustrating limits of shallow observations, while outlining a systems analysis that distinguishes query‑head unions, physical page allocation, and KV‑transfer payload to provide an arithmetic break‑even condition for pruning.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.