arXiv Machine Learning
Procedural Core: A Compact Recurrent Initialization for Vision Transformers
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 29
The Context-Ready Transformer
arXiv:2606. 27538v1 Announce Type: cross Abstract: We introduce the context-ready transformer, a new recurrent neural network architecture built from a D-layer transformer block that pre-contextualizes each token before it enters the block.
arXiv Machine Learning
Jul 2
Soft Mixture-of-Recursions: Going Deeper with Recursive Vision Transformers
arXiv:2607. 00774v1 Announce Type: cross Abstract: Recent recursive Transformer studies have primarily reused shared parameters across computation steps to construct compact, parameter-efficient models.
arXiv Machine Learning
Sep 10
Conditioned Initialization for Attention
arXiv:2609.07086v1 Announce Type: new Abstract: Transformers are a dominant architecture in modern machine learning, powering applications across vision, language, and beyond. At the core of their su...
arXiv AI
Sep 10
Precision at Scale: Domain-Specific Datasets On-Demand
arXiv:2407.03463v2 Announce Type: replace-cross Abstract: In the realm of self-supervised learning (SSL), conventional wisdom has gravitated towards the utility of massive, general domain datasets fo...
arXiv Computer Vision
Sep 10
Let ViT Speak: Generative Language-Image Pre-training
arXiv:2605.00809v3 Announce Type: replace Abstract: In this paper, we present \textbf{Gen}erative \textbf{L}anguage-\textbf{I}mage \textbf{P}re-training (GenLIP), a minimalist generative pretraining...
arXiv Computer Vision
2d ago
Do Vision Transformers Need All-to-All Attention? Global Communication Through Elastic Learned Cores
arXiv:2605.12491v2 Announce Type: replace Abstract: Vision Transformers (ViTs) learn rich visual-semantic representations through all-to-all self-attention among patch tokens. However, this design im...