arXiv:2607. 17624v1 Announce Type: new Abstract: Transformers are remarkably versatile and their design is largely consistent across a variety of applications.
arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.
arXiv:2604.13082v3 Announce Type: replace-cross Abstract: Large language models today still make mistakes on multi-digit arithmetic when they do not have tools, and part of the reason is how numbers...
arXiv:2608. 00355v1 Announce Type: cross Abstract: Progress in large language models is often summarized using a single scalar measure, such as a time horizon, a latent ability estimate, or an aggregate benchmark score.
arXiv:2608. 03921v2 Announce Type: replace Abstract: This paper offers a new interpretation of the Transformer during inference.
arXiv:2605. 22488v2 Announce Type: replace Abstract: Structured prompts require integrating components according to task-relevant relations.
The paper investigates whether removing a small set of attention heads in transformer-based language models can improve task performance, a phenomenon termed Capability Enhancement Localization (CEL). Empirical experiments across various models identify sparse head combinations whose ablation boosts performance without fine-tuning. The authors also provide a theoretical explanation using a simplified attention model, showing that training dynamics can concentrate conflicting task features into specific heads, whose removal selectively benefits one task while harming another.
arXiv:2608. 03921v1 Announce Type: new Abstract: This paper offers a new interpretation of the Transformer during inference.
arXiv:2610.14005v1 Announce Type: new Abstract: Under the accelerated pace of AI development, any benchmark whose difficulty is calibrated with humans or current AI systems becomes circular, or satur...
arXiv:2607. 01571v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning enables large language models (LLMs) to solve complex problems by generating intermediate reasoning steps.
arXiv:2605. 09724v2 Announce Type: replace Abstract: Existing accounts of grokking explain the phenomena in terms of mechanistic frameworks such as circuit efficiency or lazy-to-rich transitions.
arXiv:2610. 34553v1 Announce Type: new Abstract: How efficiently can transformers store factual knowledge?