arXiv:2606. 01172v1 Announce Type: new Abstract: Modeling unknown latent functions from finite, irregularly sampled measurements is a recurring challenge across science and engineering.
By Peiman Mohseni, Nick Duffield, Raymond K. W. Wong
LoopVAE introduces a recurrent depth architecture that reuses a scale‑ and loop‑conditioned core across different spatial scales while keeping resolution‑changing transitions separate. The four‑block core applies 28 block operations per encoder or decoder, enabling a 29M‑parameter convolutional model to achieve 0.28 rFID and 32.54 dB PSNR on ImageNet‑256 with roughly 65% fewer parameters than comparable VAEs. Experiments with both convolutional and Transformer operators, as well as ablations on parameter sharing, demonstrate competitive image quality metrics and reveal how targeted loop interventions and truncation affect reconstruction quality and computational trade‑offs.
By Zhiying Lu
The paper investigates how different attention mechanisms affect the performance of DeepONet neural operators. Five variants—varying in cross‑attention, self‑attention, tokenization, and attention depth—are trained in both data‑driven and physics‑informed settings on one‑ and two‑dimensional PDE benchmarks. Results show that per‑sensor tokenization with cross‑attention consistently reduces error, while branch self‑attention helps only in complex spatial problems, and deeper cross‑attention yields diminishing returns with higher cost.
By Amar Alem Koric, Qibang Liu, Seid Koric
arXiv:2606. 00746v1 Announce Type: cross Abstract: Vision foundation models are bottlenecked by the quadratic cost of self-attention, which limits usable resolution and increases the cost of large-scale pretraining.
By Yitong Jiang, Hongjun Wang, Collin McCarthy, Hanrong Ye, David Wehr, Xinhao Li, Qi Dou, Tianfan Xue, Ka Chun Cheung, Simon See, Wonmin Byeon, Ke Chen, Kai Han, Jinwei Gu, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Sifei Liu
The paper introduces Gated Token Recurrence (GTR), a softmax‑free recurrent vision backbone that replaces global softmax attention with gated linear attention, alternating scan directions, and enhanced SwiGLU blocks. GTR is distilled from a DINOv3 teacher using only final‑layer patch‑token alignment, and achieves strong performance on COCO object detection (58.9 box AP) with very low latency (1.908 ms on an RTX 4090). The backbone also transfers to multiple dense prediction tasks and runs efficiently on edge hardware via a specialized CUDA operator and TensorRT deployment.
By Zhe Feng, Longfei Liu, Wei Liu, Kai Chen, Jiangjiang Kong, Wei Zhou, Yifeng Qian, Dexiong Chen, Xuanlong Yu, Xi Shen
The paper introduces two multi-stage neural operator learning frameworks—Deep Collocation Neural Operator (DCNO) and Deep Galerkin Neural Operator (DGNO)—for efficiently computing convolution integrals. DCNO is a supervised method that iteratively refines operator approximations by learning residuals from data pairs, while DGNO is an unsupervised approach that uses the weak form of a PDE residual when the operator can be represented by a PDE. Both frameworks build basis operators across multiple training stages, yielding markedly higher accuracy than one-shot learning and achieving near machine‑precision results for convolution problems, with significant efficiency gains for repeated queries or parametric variations.
By Zhiping Mao, Zhenye Wen, Yong Zhang, Xiaofei Zhao