Generic object locating and segmentation based on deep convolutional neural networks and level-set methods

Abstract

The problem of object locating and segmentation within daily images is closely related to many computer vision and image editing applications, such as object detection and tracking, context-based image retrieval, and data-driven scene synthesis, etc. In this thesis, the topic is discussed regarding two important techniques: quality assessment of object proposals and automatic segmentation of foreground objects.

The first part of the thesis presents a generic pipeline for high-quality object discovery over internet images. The pipeline is built around dense proposal generation and object quality assessment by deep convolutional neural networks. The proposals are generated using state-of-the-art methods, and are further re-ranked by a quality assessment network. The network takes a given image and its object proposals as the input, and outputs quality scores, with high values indicating good quality and low values for bad quality. In this work, the concepts of completeness and fullness are introduced as major criteria for quality assessment, and are used in training the network. It is shown, through extensive experiments, the performance of existing object proposal generators can be significantly improved by re-ranking their generated proposals. It is also evident that a good combination of both region and edge features extracted from pre-trained deep convolutional neural networks makes quality assessment more reliable, compared to traditional methods using features from a single network.

In the second part of the thesis, automatic foreground object segmentation is discussed. Given good-quality object proposal windows, the segmentation of foreground objects within them can be automated with the help of a high-performance saliency detector and carefully designed segmentation procedures. The experiments conducted in this work suggest that saliency maps, as strong clues for the locations of foreground objects, can be used to initialize the segmentation, removing the need for tedious user interaction. A multi-pass level-set method based on multi-scale region and boundary features are proposed for overcoming possible initialization inaccuracy and obtaining acceptable segmentation results.