SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints

Abstract

<p>Semantic segmentation of remote sensing imagery plays a pivotal role in extracting precise information for diverse downstream applications. Recent development of the segment anything model (SAM), an advanced general-purpose segmentation model, has revolutionized this field, presenting new avenues for accurate and efficient segmentation. However, SAM is limited to generating segmentation results without class information. Meanwhile, the segmentation map predicted by current methods generally exhibits excessive fragmentation and inaccuracy of boundary. This article introduces a streamlined framework designed to leverage the raw output of SAM by exploiting two novel concepts called SAM-generated object (SGO) and SAM-generated boundary (SGB). More specifically, we propose a novel object consistency loss and further introduce a boundary preservation loss in this work. Considering the content characteristics of SGO, we introduce the concept of object consistency to leverage segmented regions lacking semantic information. By imposing constraints on the consistency of predicted values within objects, the object consistency loss aims to enhance semantic segmentation performance. Furthermore, the boundary preservation loss capitalizes on the distinctive features of SGB by directing the model's attention to the boundary information of the object. Experimental results on two well-known datasets, ISPRS Vaihingen and LoveDA Urban, demonstrate the effectiveness and broad applicability of the proposed method. The source code for this work is accessible at https://github.com/sstary/ SSRS.</p>