BuildWin-SAM: An Improved SAM-Based Method for Extracting Building Windows From Street View Images

Abstract

<p>Building facade segmentation provides critical support for urban information management, precise 3D reconstruction, and energy consumption analysis. Window, as a pivotal component of building facades, plays a central role in these applications. However, accurately identifying windows in diverse urban environments poses significant challenges due to dataset limitations and variability in model performance. This study presents two primary contributions: first, we develop the Street View Building Window (SVBW) segmentation dataset, comprising 1,172 images that represent diverse urban contexts and window types, with a total of 50,321 meticulously annotated window instances. This dataset addresses existing gaps in segmenting irregular building facades. Second, we propose BuildWin-SAM, a model for window extraction based on the Segment Anything Model (SAM) architecture, which is trained on the SVBW dataset. Comparative analysis with CNN-based semantic segmentation models and SAM demonstrates that BuildWin-SAM achieves improvements across key evaluation metrics, including Intersection over Union (IoU), F1 score, precision, and recall. Specifically, BuildWin-SAM achieves an IoU of 80.70%, precision of 89.43%, recall of 89.20%, and an F1 score of 88.52%, demonstrating precise window localization and delineation capabilities. To further validate its robustness, we conduct evaluations on three public datasets featuring multi-scale and multi-scene images with building window annotations. BuildWin-SAM achieves Recall rates exceeding 72% and Precision rates mainly above 87% across these datasets. These results demonstrate BuildWin-SAM's potential to significantly enhance building window recognition in diverse urban environments, ultimately contributing to advancements in building information management and other relevant applications. The SVBW dataset will be provided at https://github.com/zhengnanle/svbw.</p>