The paper presents a block‑wise differentiable Sinkhorn attention mechanism designed for long‑context balanced entropic optimal transport on TPU hardware. By stopping a $T$‑step Sinkhorn solve and unrolling a short refinement tail, the authors derive an exact surrogate gradient that achieves efficient block‑wise cost and memory usage. Experimental results on synthetic masked problems and a Pfam protein‑family screen demonstrate high numerical accuracy and sustained throughput on TPU v6e‑8, with notable improvements in reconstruction and sparse cross‑entropy metrics.
arXiv:2607. 24741v2 Announce Type: replace-cross Abstract: Positive two-marginal entropic optimal transport is solved by a nonlinear, positive, order-preserving, homogeneous Sinkhorn map.
arXiv:2607. 20594v1 Announce Type: cross Abstract: When does a weight-tied looped transformer -- one block applied T times -- implement an actual algorithm?
arXiv:2608. 06762v1 Announce Type: new Abstract: Bisimulation metrics quantify behavioral similarity in Markov decision processes, but their Wasserstein fixed-point operator updates every state pair and incurs quadratic pairwise work.
arXiv:2608.30427v1 Announce Type: cross Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, pres...
arXiv:2608. 05863v1 Announce Type: new Abstract: Modern models no longer keep a plain KV cache: latent caches, learned sparse selectors and recurrent states each carry the model's memory in a different form, and each fails differently under compression.
arXiv:2607. 14568v1 Announce Type: cross Abstract: A companion study ran a 35B mixture-of-experts model on a 2011 NVIDIA Tesla C2075 (Fermi, sm_20, 6GB) as a GPU-prefill/CPU-decode hybrid, because the 4-bit model did not fit in device memory (arXiv:2606.
arXiv:2609.01129v1 Announce Type: new Abstract: We identify a recurrent algebraic regularity in Transformer attention: a sparse subset of effective OV operators $T=OV^\top$ nearly closes under compos...
TwinKV is a training‑free, attention‑free repair pass that identifies and swaps orphaned and redundant tokens in a KV cache, improving long‑context inference for small models. It works by detecting near‑duplicate keys and can be composed with existing eviction policies without altering their scoring rules. Experiments on Qwen3‑4B and Llama‑3.2‑1B across LongBench, LooGLE, RULER, and MMLU‑Pro show that TwinKV consistently improves performance for most configurations, especially at tighter compression ratios.
DrainSinkhorn is a verifier‑gated active‑packing layer that improves batched entropic optimal transport (EOT) by eliminating finished problems from subsequent Sinkhorn updates. It combines candidate‑axis packing, a one‑sided screen, verifier‑gated retirement, and physical compaction, while keeping the EOT objective, per‑instance map, and stopping rule unchanged. The method achieves state‑of‑the‑art execution speedups—up to 4.11× faster on MetroPT‑3 and 3.80× on ImageNet‑32 feature couplings—across multiple backends and tolerance settings. whyItMatters":"The technique delivers significant runtime reductions for heterogeneous batched‑EOT workloads, enabling faster and more efficient optimal transport computations in practical machine‑learning pipelines."
arXiv:2608. 11318v1 Announce Type: cross Abstract: Many sequential construction tasks exhibit exact symmetry at completion while their execution remains directed and history-dependent.
arXiv:2608. 03447v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel.
MemoryWalker addresses the conditioning problem that arises when training agents with compressed context during rollout. It introduces two exact, gradient‑equivalent corrections—LogitTree, a segmented K‑forward traversal, and a packed 4D attention mask—alongside SDCC, a self‑distillation method that reduces the train‑deployment gap by minimizing KL divergence at each eviction. Experiments on seven web‑search benchmarks show that SDCC significantly lowers logit drift and boosts rollout rewards compared to naive training.