arXiv Machine Learning

NAIMA: Semantics Aware RGB Guided Depth Super-Resolution

The paper introduces NAIMA, a guided depth super‑resolution framework that leverages global contextual semantic priors from pretrained vision transformer token embeddings. Its Guided Token Attention (GTA) module uses depth encodings as queries to attend over semantic tokens, with a zero‑initialized gate controlling the influence of semantic evidence. NAIMA achieves competitive in‑distribution performance while delivering superior cross‑dataset generalization without relying on decoded priors or auxiliary objectives.

arXiv Machine Learning

Aug 27

WAVE: Reversing the Guidance Hierarchy for Coarse-to-Fine Guided Depth Super-Resolution

WAVE introduces a multi-level discrete wavelet transform (ML‑DWT) to reverse the typical fine‑to‑coarse bias in guided depth super‑resolution. By consuming wavelet sub‑bands and semantic tokens in reverse order, it separates structure and detail reconstruction, applies semantic gating to high‑frequency bands, and fuses modalities via an invertible coupling mechanism. Experiments on multiple benchmarks show that WAVE matches or outperforms existing methods, especially at high upsampling factors where low‑resolution depth has minimal structure.

By Tayyab Nasir, Daochang Liu, Ajmal Mian

benchmarks safety

arXiv AI

Jul 7

MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources

arXiv:2601. 22054v2 Announce Type: replace-cross Abstract: Scaling has powered recent advances in vision foundation models, yet extending this paradigm to metric depth estimation remains challenging due to heterogeneous sensor noise, camera-dependent biases, and metric ambiguity in noisy cross-source 3D data.

By Baorui Ma, Jiahui Yang, Donglin Di, Xuancheng Zhang, Jianxun Cui, Hao Li, Yan Xie, Wei Chen

llms computer-vision multimodal benchmarks

arXiv AI

4d ago

OpenDepth: Repurposing Image Editing Priors for Robust Open-World Monocular Depth Estimation

arXiv:2610.17610v2 Announce Type: new Abstract: Recently, the integration of generative priors has revolutionized Monocular Depth Estimation (MDE), achieving state-of-the-art (SOTA) performance with...

By Mengke Zhang, Tianyi Yang, Xinzhuo Li, Anlong Ming

llms diffusion computer-vision benchmarks safety

arXiv AI

Jun 15

Vanishing Depth: Training Generalized Depth Adapters with Sinusoidal Depth Preprocessing for Pretrained RGB Encoders

arXiv:2503. 19947v2 Announce Type: replace-cross Abstract: Generalized metric depth understanding is critical for precise vision-guided robotics, which current state-of-the-art (SOTA) vision-encoders do not support.

By Paul Koch, J\"org Kr\"uger

diffusion computer-vision robotics fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 1

Fast and Accurate Image Restoration and Generation with Rank Enhanced Linear Attention

arXiv:2505.16157v3 Announce Type: replace Abstract: Transformer-based models have made remarkable progress in image restoration (IR) tasks. However, the quadratic complexity of self-attention in Tran...

By Yuang Ai

llms diffusion computer-vision benchmarks

arXiv Computer Vision

Oct 2

Do Vision Transformers Need All-to-All Attention? Global Communication Through Elastic Learned Cores

arXiv:2605.12491v2 Announce Type: replace Abstract: Vision Transformers (ViTs) learn rich visual-semantic representations through all-to-all self-attention among patch tokens. However, this design im...

By Alan Z. Song, Yinjie Chen, Mu Nan, Deva Ramanan, Michael J. Tarr, Andrew F. Luo

llms computer-vision benchmarks

arXiv AI

Aug 11

UniSpace: Unified Visual Representation and Scalable Multimodal Modeling

arXiv:2608. 08676v1 Announce Type: cross Abstract: Semantic vision encoders have become a central visual interface for multimodal understanding and semantic conditioning in image generation.

By Jinbo Yan, Limeng Qiao, Jie Qin, Junyan He, Feize Wu, Guanglu Wan

llms rag diffusion multimodal

arXiv AI

Jun 26

Semantic Generative Tuning for Unified Multimodal Models

arXiv:2605. 18714v2 Announce Type: replace-cross Abstract: Unified multimodal models (UMMs) strive to consolidate visual understanding and visual generation within a single architecture.

By Songsong Yu, Yuxin Chen, Ying Shan, Yanwei Li

computer-vision multimodal benchmarks

arXiv Computer Vision

Sep 22

RefAM: Attention Magnets for Zero-Shot Referral Segmentation

arXiv:2509.22650v3 Announce Type: replace Abstract: Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models,...

By Anna Kukleva, Enis Simsar, Alessio Tonioni, Muhammad Ferjad Naeem, Federico Tombari, Jan Eric Lenssen, Bernt Schiele

llms diffusion computer-vision fine-tuning multimodal benchmarks

arXiv AI

Jul 10

LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windows

arXiv:2604. 05182v2 Announce Type: replace-cross Abstract: We introduce the Large Sparse Reconstruction Model to study how scaling transformer context windows affects feed-forward 3D reconstruction.

By Zhengqin Li, Cheng Zhang, Jakob Engel, Zhao Dong

llms benchmarks

arXiv Computer Vision

Sep 1

Efficient and High-Quality Depth Estimation via Pixel-Space Diffusion with Linear Attention

arXiv:2608.30129v1 Announce Type: new Abstract: This work presents $\textbf{Lapis}$, a $\textbf{l}$inear-$\textbf{a}$ttention-based $\textbf{pi}$xel-$\textbf{s}$pace generative framework that achieve...

By Bingde Liu, Wu Ran, Jinglei Zhang, Huanhuan Yuan, Chao Ma

diffusion computer-vision efficiency benchmarks

arXiv AI

Jun 6

Image Generators are Generalist Vision Learners

arXiv:2604. 20329v3 Announce Type: replace-cross Abstract: Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining.

By Valentin Gabeur, Shangbang Long, Songyou Peng, Paul Voigtlaender, Shuyang Sun, Yanan Bao, Karen Truong, Zhicheng Wang, Wenlei Zhou, Jonathan T. Barron, Kyle Genova, Nithish Kannen, Sherry Ben, Yandong Li, Mandy Guo, Suhas Yogin, Yiming Gu, Huizhong Chen, Oliver Wang, Saining Xie, Howard Zhou, Kaiming He, Thomas Funkhouser, Jean-Baptiste Alayrac, Radu Soricut

llms diffusion computer-vision benchmarks

Related stories