What Accuracy Hides In Learned Transformer Programs

The paper investigates Transformer Programs (TP), a constrained Transformer architecture that outputs readable Python code. It shows that token‑level accuracy can mask incorrect full‑sequence outputs, and that inspecting the generated programs reveals failures missed by standard tests. By using program inspection to guide data augmentation, the authors achieved higher sequence accuracy on several algorithmic tasks, demonstrating the value of program‑level analysis for training TP models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms