arXiv Computer Vision

Domain Adaptation of SAM 2.1 for Building-Component Segmentation in Image-Based 3D Reconstruction

The paper reports fine‑tuning the Segment Anything Model (SAM) 2.1 on the HBD dataset of 2,679 images from 42 buildings to improve multi‑class segmentation of 2D building images. The fine‑tuned model achieves a higher mean Intersection‑over‑Union (mIoU) of 69.6% compared to 49.9% for the pretrained version, and shows better performance on larger object classes. These enhanced 2D masks are then used in Group Anything with Radiance Fields (GARField) to group building components more accurately in 3D neural radiance fields, facilitating more precise 3D auto‑reconstruction of building structures.

arXiv Computer Vision
Sep 18

Instance Segmentation and Fine-grained Classification for Urban Buildings with Adaptive Region Dividing and Spatially-Supervised Contrastive Learning

The paper introduces an adaptive region‑dividing strategy that projects a 3D point cloud onto a bird’s‑eye‑view plane to detect building regions, then back‑projects bounding boxes to create structure‑aligned training blocks for unified scene‑level evaluation. It also proposes a fine‑grained classification model using a point transformer classifier and a spatially‑supervised contrastive loss to improve inter‑class discriminability, addressing class imbalance with a weighted cross‑entropy. Experiments on UrbanBIS and STPLS3D datasets show the method outperforms state‑of‑the‑art approaches in both building instance segmentation and fine‑grained classification.

By Weiyuan Zhang, Qi Zhang, Hui Huang
arXiv Computer Vision
Oct 12

Measurable Semantic 3D Gaussian Splatting with Joint Region and Depth Supervision for Interactive Building Inspection

The paper introduces a measurable semantic 3D Gaussian Splatting (3DGS) framework that uses region and depth supervision to improve building inspection from multi‑view imagery. Mask‑based region supervision focuses optimization on target areas, while point‑wise depth supervision aligns Gaussian centers with physical surfaces derived from multi‑view stereo point clouds. The resulting semantic Gaussian field enables component‑level inspection and is deployed in the AEC‑SplatTwin platform, achieving significant reductions in depth error and high semantic segmentation accuracy on UAV‑inspected and smartphone‑captured datasets.

By Yi Zhang, Tian Tan, Ray Kai Leung Su
arXiv Computer Vision
Sep 1

SVI2LoD3: Agent-Driven Reconstruction of LoD3 Facade Openings in Semantic 3D City Models from Volunteered Street View Imagery using Large Language and Visual Models

arXiv:2608.29992v1 Announce Type: new Abstract: This paper presents an end-to-end, agent-driven pipeline for the LoD3 reconstruction of facade openings in 3D city models, producing directly usable Ci...

By Elmehdi Kanna, Lukas Arzoumanidis, Huynh Duc An Son Nguyen, Youness Dehbi
arXiv Machine Learning
5d ago

Vision Transformer Ensembles for Panoramic Street Segmentation

The paper presents a two‑stage approach to street‑panorama semantic segmentation using Vision Transformer ensembles. In the first stage, nine pretrained models are compared under equal compute budgets, and the best two are trained with three random seeds, achieving 60.95% mIoU on validation and 57.08% on the hidden test leaderboard. The second stage explores augmentations, larger inputs, and additional architectures, culminating in an ensemble of four models that scores 64.53% mIoU on validation and 61.54% mIoU (71.93% F1) on the hidden test leaderboard, while the study documents all runs, costs, and releases the code and models.

By Yunus Serhat B{\i}\c{c}ak\c{c}{\i}
arXiv Computer Vision
Oct 7

Building Rome from a Single Image

arXiv:2610.08790v1 Announce Type: new Abstract: Single-image scene generation aims to produce a complete 3D scene mesh from a single image, including surfaces the camera did not observe. While pretra...

By Jiraphon Yenphraphai, Fang Li, Tianshuo Xu, Depu Meng, Quentin Herau, Yihan Hu, Raymond A. Yeh, Wei Zhan
arXiv Computer Vision
Oct 16

The First Assessment of PhiSat-2 Imagery for Monocular Building Height Estimation

PhiSat‑2 offers 4.75 m multispectral imagery that can be used for monocular building‑height estimation. The authors created the PhiSat‑2–Height Dataset (PHDataset), comprising 9,475 co‑registered patch pairs from 26 cities, and developed the Two‑Stream Ordinal Network (TSONet) that uses multiscale decoupled height representation and footprint‑conditioned height reconstruction. In city‑disjoint tests, TSONet improves building RMSE and MAE by 3.64 % and 4.49 % over the best existing method, and PhiSat‑2 outperforms Sentinel‑2 and Landsat 8, demonstrating its value for urban height mapping.

By Yanjiao Song, Bowen Cai, Timo Balz, Zhenfeng Shao, Neema Simon Sumari, James Magidi, Walter Musakwa
arXiv Computer Vision
Sep 22

Semi-automated reconstruction of indoor geometry from 360-degree video for CFD-based airflow analysis in classrooms

The paper presents a semi‑automated pipeline that transforms a single 360‑degree video of a classroom into editable, simulation‑ready 3D geometry for CFD analysis. Using NeRF for dense point clouds, SAM‑based 2D instance masks lifted to 3D, and octree‑based instance separation, the workflow produces object‑level assets that can be reconfigured in a browser editor. The resulting geometry is validated with OpenFOAM simulations and benchmarked against IEA Annex 20, demonstrating that per‑room geometry acquisition is essential for accurate airflow predictions.

By Dhruv Gamdha, James Afful, Shambhavi Joshi, Ulrike Passe, Adarsh Krishnamurthy, Baskar Ganapathysubramanian