Algebraic Invariants of Lightning Self-Attention
Read the original on arXiv Statistics ML →The paper investigates the polynomial coefficients of lightning self‑attention, treating them as coordinates of an algebraic variety. In the single‑token case it identifies the coefficient variety as a rank‑constrained Chow‑type variety and derives algebraic equations; for multiple tokens it shows that linear relations reduce the geometry to coefficients involving interactions between distinct tokens, characterized by a common linear factor and a low‑rank condition. The authors provide explicit families of determinantal, Veronese‑type, and Sylvester resultant‑based invariants, and in the rank‑one case give pencil and flattening equations that define the variety set‑theoretically, with small‑dimension computations confirming the theoretical generators.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Statistics ML.