Small Enough to Know Everything: The Fully-Enumerable Transformer as an Instrument for the Science of Delayed Generalization
Read the original on arXiv Machine Learning →The paper introduces the fully‑enumerable transformer—a tiny transformer trained on tasks where every input can be evaluated exactly—as a scientific instrument for studying delayed generalization. It claims four unique capabilities: exact, falsifiable generalization ceilings; precise task surgery; direct observation of all weights; and survival‑time statistics that treat non‑grokking as censored data. A preregistered conservation study shows that two task‑side laws (a recoverability‑ceiling law and a role‑conflict delay law) hold across a 4,000‑fold increase in model size, while a weight‑decay law deforms predictably, demonstrating that the instrument can reveal lawful patterns of generalization across scales.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.