Medical image segmentation jointly using data-driven method and model-based method

Abstract

The repaid growth of medical images brings a vast work burden for radiologists and physicians. Image segmentation, i.e., assigning every single voxel a category, is a critical problem in many applications, such as detecting liver tumors or volumetric analysis of stroke lesions. This study investigates how to obtain more accurate segmentation results from 3D medical images.

With the availability of large amounts of imaging data, significant progress has been made in medical image segmentation using data-driven methods, in particular deep learning. However, purely data-driven methods sometimes may fail. Some model-based methods may fix the errors. To improve the clinical image segmentation's efficiency and robustness, we apply some model-based methods before and after the neural network module, i.e., pre-processing and post-processing.

This thesis starts with a coarse-fine-refine framework designed for a small organ segmentation task from computed tomography (CT) images. As a kind of small organ, pancreas occupies only a tiny part of the whole CT images, and this highly unbalanced class distribution may cause bias for the deep learning-based model. Given that a deep learning model using a smaller input region can obtain a more accurate segmentation, we train the model only using the smaller region around the region of interest (ROI). At the testing stage, we use an atlas-based method to localize the pancreas in the pre-processing step and then use the deep learning-based model to predict the pancreas within the ROI. This strategy is named coarse-to-fine. Finally, we apply a level-set method (LSM) to refine the deep learning-based segmentation results.

Following that, we extensively apply the coarse-to-fine strategy and the LSM-based post-processing to a lesion segmentation task from CT images. We use the coarse-to-fine strategy to obtain a more accurate liver segmentation and then serve it as the mask to remove non-liver tissues. We further use the LSM to repair the unexplainable UNet-based liver tumor segmentation. This work shows the proposed framework's potential in lesion segmentation. Besides, we design an adaptive intensity truncation in the pre-pressing step to enhance the image contrast between lesion and organ.

After that, we further investigate the importance of prior localization information in brain lesion segmentation tasks from magnetic resonance images. Brain parcellation, dividing the human brain into different sub-regions according to their function, is important prior localization information. We find that brain parcellation can improve stroke lesion segmentation accuracy.

In summary, these studies have demonstrated that model-based methods can make data-driven methods to avoid unexplainable results. Data-driven methods can provide automatic feature extraction for model-based methods. On this basis, data-driven methods and model-based methods should be jointly considered when designing a segmentation pipeline.