arXiv:2607. 03612v1 Announce Type: cross Abstract: Feed-forward 3D reconstruction (F3R) transformers have recently achieved remarkable success.
arXiv:2608. 10519v2 Announce Type: replace Abstract: InfinityStar extends visual autoregressive generation to video through a sequence of image and clip pyramids.
arXiv:2609.23733v1 Announce Type: new Abstract: Feed-forward visual geometry models such as the Visual Geometry Grounded Transformer (VGGT) have recently enabled direct 3D reconstruction from multi-v...
Visual Geometry Grounded Transformer (VGGT) recovers dense 3D scene structure from multi-view images in one forward pass, but quadratic cross-frame attention limits its scalability. Existing training-free accelerators reduce computation uniformly along one axis, missing layer heterogeneity.
InfinityStar extends visual autoregressive generation to video through a sequence of image and clip pyramids. Its changing scale and cross-clip context, however, leave late-scale attention costly and make sparse patterns reused from diffusion or image VAR models unreliable.
LoG-VGGT is a memory‑efficient framework for long‑sequence 3D reconstruction that balances local temporal modeling with global camera consistency. It uses cross‑window attention in a small subset of transformer blocks to propagate information across adjacent temporal windows while keeping memory usage bounded. A global camera consistency refinement module further improves long‑horizon pose stability by enforcing scene‑level constraints through cross‑attention between camera and compact register tokens, leading to better depth accuracy and robust camera pose estimation on multiple benchmarks.
The paper introduces SparsePR, a training‑free block‑sparse attention method for video transformers that partitions query‑key responses and reconstructs the residual via probe‑fitted affine corrections. By pairing sampled‑query key responses into K/V groups and using centroids to guide shared routing, SparsePR reduces attention‑reconstruction error across diverse video generation and world‑model tasks. Experiments show consistent error reductions, with probe fitting contributing most of the improvement, while maintaining generation quality at 22.0–26.0% executed‑pair density and delivering 1.48×–2.61× speedups.
arXiv:2605.12491v2 Announce Type: replace Abstract: Vision Transformers (ViTs) learn rich visual-semantic representations through all-to-all self-attention among patch tokens. However, this design im...
arXiv:2606. 06467v1 Announce Type: cross Abstract: Long-context inference in modern LLMs is increasingly constrained by decoding efficiency, especially in reasoning-heavy settings where models generate long intermediate chains of thought.
The paper investigates whether deep transformer layers require context from the residual stream to compute value vectors. It finds that allowing deeper layers to use a context‑free value vector—preserving original token information—significantly improves performance, and adding context afterward yields little extra benefit. The authors introduce Bank of Values (BoV), a lookup table of token‑specific value vectors for the last third of layers, which reduces compute and memory while matching or surpassing prior methods on large models.
LayerRecall is a memory router for autoregressive video diffusion that selectively retrieves and injects historical key/value states into specific layers of the model, based on the current context. It addresses the problem that existing memory mechanisms expose nonlocal history but do not guarantee effective use, by recognizing that different layers prefer current, recent, or distant context. The method, combined with Cross‑Horizon Prediction Matching, achieves state‑of‑the‑art long‑range consistency on MemoBench and MovieBench while maintaining local continuity and incurring negligible inference overhead.
arXiv:2604. 05182v2 Announce Type: replace-cross Abstract: We introduce the Large Sparse Reconstruction Model to study how scaling transformer context windows affects feed-forward 3D reconstruction.