arXiv:2609.25438v1 Announce Type: new
Abstract: Diverse pretraining has been shown to be an effective method for learning reusable, domain-aware representations that provide a starting point for fine...
By Henry Kvinge
arXiv:2604. 07822v2 Announce Type: replace-cross Abstract: We study implicit reasoning, i.
By Harsh Kohli, Srinivasan Parthasarathy, Huan Sun, Yuekun Yao
The paper introduces graph metanetworks for transformer-based language models, addressing the challenge that different weight configurations can implement the same function due to a rich symmetry group. It formalizes a weight‑space symmetry group for complete decoder‑only language models and presents a graph construction invariant to these symmetries. The authors also create two new transformer‑population datasets with tasks like training‑corpus prediction and fine‑tuning detection, where their method outperforms existing baselines.
By Yonathan Wolanowsky, Guy Bar-Shalom, Fabrizio Frasca, Haggai Maron
arXiv:2505. 20161v2 Announce Type: replace-cross Abstract: Effective generalization in language models depends critically on the diversity of their training data.
By Jaehun Jung, Seungju Han, Ximing Lu, Skyler Hallinan, David Acuna, Shrimai Prabhumoye, Mostafa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
arXiv:2608. 08113v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has become the dominant paradigm for eliciting reasoning in Large Language Models (LLMs), yet it creates substantial computational overhead by forcing models to externalize intermediate reasoning steps as discrete tokens.
By Abhishek Panwar, Maheep Singh, Saksham Bansal
arXiv:2510. 25013v2 Announce Type: replace-cross Abstract: Mechanistic interpretability aims to reverse-engineer large language models (LLMs) into human-understandable computational circuits.
By Rabin Adhikari