arXiv Machine Learning
Increasing Skill Level Recruits Deeper Attention Layers in a Frozen Chess Transformer
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
6d ago
Query Expansion and Key Specialization in Transformer Attention Geometry
The projection of queries and keys are central to the attention mechanism in Transformer architectures. While they are mathematically symmetric, they play different roles in attention mechanisms.
arXiv Machine Learning
Sep 2
Performance-Efficiency Tradeoffs in Transformers: An Approximation Theory Perspective
The paper examines how to allocate attention heads and head dimensions across Transformer layers to balance expressivity and efficiency. It provides a mathematical analysis of early layers’ role in information extraction and characterizes the trade‑off between head count and dimension under a fixed parameter budget. The authors prove a saturation effect of softmax activations, showing that increasing head dimensions yields diminishing returns, especially for long sequences, and propose strategies for efficient parameter allocation across layers.
arXiv AI
Jul 15
Matilda: Engine-Agnostic Search with Human Policy Guidance
arXiv:2606. 25176v2 Announce Type: replace Abstract: Chess engines have evolved from search-based systems optimized solely for strength to neural policies capable of modeling human decisions across much of the rating spectrum.
arXiv AI
Jun 24
More Skills, Worse Agents? Skill Shadowing Degrades Performance When Expanding Skill Libraries
arXiv:2605. 24050v2 Announce Type: replace-cross Abstract: Skill libraries allow LLM agents to load task-specific instructions on demand, letting non-expert users solve domain-specific tasks through natural language without knowing which skills exist or how they work.
arXiv AI
Jul 22
A Controlled Study of Attention-Only Transformers
arXiv:2607. 18363v1 Announce Type: cross Abstract: Feed-forward networks hold two thirds of a transformer's non-embedding parameters, yet the architecture has not received a necessity test that controls parameters, compute, and depth at once.
arXiv AI
Jun 26
Accelerating Skill Assessment in Chess: A Drift-Diffusion-Enhanced Elo Rating System
arXiv:2606. 26267v1 Announce Type: new Abstract: Rating systems such as Elo serve as the gold standard for matchmaking in competitive chess.