arXiv Computer Vision

DIFTA-3D: Depth-Consistent Instance-Level Feature Transfer and Adaptation of DINOv3 for 3D Detection

The paper introduces DIFTA-3D, a method that replaces the task‑specific visual branch in IIFNet3D with a frozen DINOv3 foundation model for RGB‑D 3D instance detection. It employs a depth‑consistent feature pipeline that projects points into calibrated RGB‑D frames, filters features with a metric depth‑residual check, caches accepted DINOv3 features, and aggregates them within proposal‑aligned RoI grids. Extensive experiments on ScanNetV2 show that the DINOv3 control achieves mAP scores of 76.15/60.93 at IoU thresholds 0.25/0.50, while the Conservative VAID recipe improves these to 76.59/62.16, indicating a modest gain from the proposed transfer recipe.

arXiv Machine Learning
Oct 9

One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts

The paper introduces reViT, a recurrent vision transformer that uses a single Transformer block repeatedly to achieve the accuracy of a full-depth encoder while keeping inference FLOPs comparable. It achieves depth‑specific transformations by expressing each block’s feed‑forward network as a convex combination of a small shared expert bank, with a continuous depth coordinate guiding the mixture. Experiments on ImageNet‑1k and DINOv2 distillation show that reViT matches or surpasses state‑of‑the‑art models, reduces stored parameters by about 70%, and supports elastic‑depth deployment from a single checkpoint.

By Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos
Hugging Face Trending Papers
4d ago

HiVE-VQA: Hierarchical Interaction via Volumetric Embedding for Video Quality Assessment

HiVE-VQA introduces a new approach to no-reference video quality assessment by leveraging a single frozen JEPA-based video foundation model. The method extracts token grids from multiple depths of a pre‑trained V-JEPA 2.1 encoder, stacks them into a hierarchical spatio‑temporal volume, and fuses them with a lightweight 3D convolutional head. With only 5.2 M trainable parameters, HiVE-VQA outperforms 17 state‑of‑the‑art methods on both intra‑dataset and cross‑dataset benchmarks, demonstrating that depth‑wise hierarchical fusion from a frozen backbone is both effective and efficient.

arXiv Computer Vision
Sep 18

SenseFuse: Label-Free Fusion of Image and Shape Encoders for Open-Vocabulary 3D Instance Segmentation

SenseFuse introduces a label‑free fusion approach that balances 2D image and 3D shape encoders for open‑vocabulary 3D instance segmentation. By selecting a scene‑level fusion weight through an adaptive, sensitivity‑based mechanism, it improves mask labeling accuracy across multiple datasets, recovering up to 93% of the potential gain from an oracle weight. The method demonstrates that image and shape encoders have complementary failure patterns, leading to higher instance AP in most evaluated settings.

By Euiseok Han, Tri Ton, Hwanhee Kim, Seungyeon Ryu, Chang D. Yoo
arXiv AI
Oct 5

OmniAct3D: Leveraging Foundation Geometry and Evidence-Grounded Reasoning for Panoramic 3D Detection

OmniAct3D is a framework that adapts Vision Foundation Model (VFM) detectors trained on perspective images to equirectangular projection (ERP) panoramic images for 3D detection. It introduces the ERP-Ray Geometry Adapter (ERGA-Ray) to model spherical rays, the Visual-Action Reasoning Chain (VARC) to ground hypotheses in panoramic evidence, and the Appearance-Guided Heading Expert (AGHE) to recover high‑resolution heading cues. The method achieves a 2.96 NDS improvement on Spheriverse and a 24.87 mAP gain on PanoMMOcc, while preserving most of the baseline’s performance across configurations.

By Runtong Wu, Fei Teng, Di Wen, Guoqiang Zhao, Kunyu Peng, Kailun Yang
arXiv AI
Sep 2

VOIM: Training-Free Open-Vocabulary 3D Instance Mapping for RGB-D and Monocular SLAM

VOIM (Voxel‑Grounded Online Instance Manager) is a training‑free system that builds open‑vocabulary 3D instance maps from RGB‑D or monocular RGB input by deferring label and instance decisions until sufficient soft evidence accumulates per voxel across views. Across four perception configurations on ScanNet++, VOIM outperforms the strongest online RGB‑D system, OVO‑SLAM, by 4.8–11.7 mIoU, and achieves 44.07 mIoU under a like‑for‑like protocol, winning all ten scenes. The method also runs unchanged on monocular RGB, matching baseline performance on Replica, and produces exportable occupancy grids that support free‑form instance queries.

By Sangmin Song, Sarath Kodagoda, Marc G. Carmichael, Karthick Thiyagarajan, Amal Gunatilake, Kelly Prentice, Jodi Martin