File Download

There are no files associated with this item.

Supplementary

Conference Paper: Open-Vocabulary Semantic Segmentation with Decoupled One-Pass Network

TitleOpen-Vocabulary Semantic Segmentation with Decoupled One-Pass Network
Authors
Issue Date2-Oct-2023
Abstract

Recently, the open-vocabulary semantic segmentation problem has attracted increasing attention and the best performing methods are based on two-stream networks: one stream for proposal mask generation and the other for segment classification using a pretrained visual-language model. However, existing two-stream methods require passing a great number of (up to a hundred) image crops into the visual-language model, which is highly inefficient. To address the problem, we propose a network that only needs a single pass through the visual-language model for each input image. Specifically, we first propose a novel network adaptation approach, termed patch severance, to restrict the harmful interference between the patch embeddings in the pre-trained visual encoder. We then propose classification anchor learning to encourage the network to spatially focus on more discriminative features for classification. Extensive experiments demonstrate that the proposed method achieves outstanding performance, surpassing state-of-the-art methods while being 4 to 7 times faster at inference.


Persistent Identifierhttp://hdl.handle.net/10722/339384

 

DC FieldValueLanguage
dc.contributor.authorHan, Cong-
dc.contributor.authorZhong, Yujie-
dc.contributor.authorLi, Dengjie-
dc.contributor.authorHan, Kai-
dc.contributor.authorMa, Lin -
dc.date.accessioned2024-03-11T10:36:10Z-
dc.date.available2024-03-11T10:36:10Z-
dc.date.issued2023-10-02-
dc.identifier.urihttp://hdl.handle.net/10722/339384-
dc.description.abstract<p>Recently, the open-vocabulary semantic segmentation problem has attracted increasing attention and the best performing methods are based on two-stream networks: one stream for proposal mask generation and the other for segment classification using a pretrained visual-language model. However, existing two-stream methods require passing a great number of (up to a hundred) image crops into the visual-language model, which is highly inefficient. To address the problem, we propose a network that only needs a single pass through the visual-language model for each input image. Specifically, we first propose a novel network adaptation approach, termed patch severance, to restrict the harmful interference between the patch embeddings in the pre-trained visual encoder. We then propose classification anchor learning to encourage the network to spatially focus on more discriminative features for classification. Extensive experiments demonstrate that the proposed method achieves outstanding performance, surpassing state-of-the-art methods while being 4 to 7 times faster at inference.</p>-
dc.languageeng-
dc.relation.ispartof2023 International Conference on Computer Vision (ICCV) (02/10/2023-06/10/2023, , , Paris)-
dc.titleOpen-Vocabulary Semantic Segmentation with Decoupled One-Pass Network-
dc.typeConference_Paper-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats