arXiv Machine Learning By Abraham Chachamovits

A Bethe-Ansatz Source of Early Token Correlations: A marginal-preserving construction for parallel proposals

Read the original on arXiv Machine Learning →

The paper constructs coordinate‑Bethe‑Ansatz joint laws for parallel token proposals using the periodic XXZ chain and compares them to maximum‑entropy pairwise projections that match the same one‑ and two‑site moments. In an eight‑site, three‑excitation sector, symmetry reduces the difference between the full probability law and its pairwise projection to a single contiguous three‑site moment δ, which changes sign and vanishes at Δ=(3+√3)/2, making the eigenstate law pairwise. Nonlinear two‑token and three‑token readouts preserve token marginals while revealing distances proportional to |δ|, with explicit deterministic binary distances computed at Δ=1.5; the authors also provide an exact target‑alignment formula and prove a strict speculative accepted‑prefix advantage in an unrestricted open neighborhood of the constructed target.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI
Oct 27

Relational Composition as Parallel Transport: What Per-Entity Frames Force in Attention

The paper investigates how treating each entity in a graph or language model as having its own coordinate frame—viewed as a symmetry—constrains the form of attention mechanisms. It shows that with orthogonal transports, invariant attention scores depend only on norms and transported inner products, while equivariant messages lie in the span of the entity vectors and their transported counterparts. The authors demonstrate that standard attention corresponds to a flat shared frame, and that incorporating per‑entity frames allows relations to be represented as operators that compose via parallel transport, yielding empirical gains on synthetic relational tasks and natural language benchmarks.

By Aakash Sarkar, Ananya Mukherjee, Motoshige Sato
arXiv Computation and Language
Aug 28

TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy

TwinKV is a training‑free, attention‑free repair pass that identifies and swaps orphaned and redundant tokens in a KV cache, improving long‑context inference for small models. It works by detecting near‑duplicate keys and can be composed with existing eviction policies without altering their scoring rules. Experiments on Qwen3‑4B and Llama‑3.2‑1B across LongBench, LooGLE, RULER, and MMLU‑Pro show that TwinKV consistently improves performance for most configurations, especially at tighter compression ratios.

By Hong Chen, Yudong Zeng, Yongwei Huang, Zuhao Ouyang, Junyan Zhang, Xuming Hu