The paper introduces Successive Capacity Growth (SCG), a method that starts with a minimal Vision Transformer encoder and incrementally expands its width or depth based on a task‑agnostic test‑and‑verify mechanism. SCG uses function‑preserving expansion and a Sketched Isotropic Gaussian Regularizer (SIGReg) to ensure independent semantic dimensions and prevent collapse. Experiments on multi‑object dynamics and 2D navigation tasks show that SCG achieves significant prediction loss reductions while being far more parameter‑efficient than fixed large models, with no false‑positive expansions and exact function preservation.
arXiv:2607. 16314v1 Announce Type: cross Abstract: World models, especially based on JEPA architectures, have been shown to learn robust dynamics of various environments.
By Usman M. Khan
arXiv:2605.12491v2 Announce Type: replace
Abstract: Vision Transformers (ViTs) learn rich visual-semantic representations through all-to-all self-attention among patch tokens. However, this design im...
By Alan Z. Song, Yinjie Chen, Mu Nan, Deva Ramanan, Michael J. Tarr, Andrew F. Luo
arXiv:2606. 16112v1 Announce Type: cross Abstract: Residual architectures are ubiquitous in deep learning, but they suffer from a subtle structural limitation: the norm of the residual stream can grow rapidly with depth.
By Tom\'as Figliolia, Beren Millidge
Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency.
arXiv:2609.23733v1 Announce Type: new
Abstract: Feed-forward visual geometry models such as the Visual Geometry Grounded Transformer (VGGT) have recently enabled direct 3D reconstruction from multi-v...
By Abteen Arab, Guile Wu, Chengjie Huang, Dongfeng Bai
The paper introduces NAIMA, a guided depth super‑resolution framework that leverages global contextual semantic priors from pretrained vision transformer token embeddings. Its Guided Token Attention (GTA) module uses depth encodings as queries to attend over semantic tokens, with a zero‑initialized gate controlling the influence of semantic evidence. NAIMA achieves competitive in‑distribution performance while delivering superior cross‑dataset generalization without relying on decoded priors or auxiliary objectives.
By Tayyab Nasir, Daochang Liu, Ajmal Mian
The paper introduces reViT, a recurrent vision transformer that uses a single Transformer block repeatedly to achieve the accuracy of a full-depth encoder while keeping inference FLOPs comparable. It achieves depth‑specific transformations by expressing each block’s feed‑forward network as a convex combination of a small shared expert bank, with a continuous depth coordinate guiding the mixture. Experiments on ImageNet‑1k and DINOv2 distillation show that reViT matches or surpasses state‑of‑the‑art models, reduces stored parameters by about 70%, and supports elastic‑depth deployment from a single checkpoint.
By Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos
The paper introduces DARTS, a method for tuning decoder representations during model merging. It addresses representation bias in autoregressive decoders by using an entropy‑weighted L1 loss and a per‑position additive bias to correct errors that accumulate across token positions. Experiments on code generation, mathematical reasoning, and instruction following with Llama‑2‑7B show that DARTS improves performance over standard surgery while adding only 0.1% extra parameters.
By Aaryan Ajay Sharma, Sai Nishanth Padala, Seganrasan Subramanian
arXiv:2605. 18848v3 Announce Type: replace Abstract: This paper introduces Exact Linear Attention (ELA), a mechanism that achieves linear computational complexity for Transformer attention by exploiting the exact decomposition property of kernel functions, thereby eliminating approximation error.
By Weinuo Ou
arXiv:2610.16817v1 Announce Type: cross
Abstract: Progressive depth growth builds large networks by starting with shallow models and inserting layers during training. Recent advances in large languag...
By Amir Mohammad Karimi Mamaghan, Ferdinand Kapl, Andrea Dittadi, Karl Henrik Johansson, Kaitlin Maile, Johannes von Oswald, Stefan Bauer
arXiv:2607. 16012v1 Announce Type: cross Abstract: Multi-Task Learning (MTL) in robotics perception systems supports comprehensive 3D spatial scene understanding by integrating semantic segmentation and depth estimation.
By Jehun Kang, Jungha Wang, Youngjun Hwang, David Hyunchul Shim