Towards effective and interpretable deep learning for biomedical image analysis

Abstract

Biomedical imaging has revolutionized medicine and biomedical research, providing non-invasive anatomical and functional information of living subjects. Massive digital image data are available in this era, exceeding human analysis capabilities. As a result, the dependence on automated algorithms becomes prominent. Recent advancements in machine learning have shown its promising potential in deciphering and understanding natural images. However, when applying machine learning to biomedical image analysis, unique challenges arise and are categorized into two streams in this thesis: effectiveness and interpretability. The challenge of effectiveness comes from the differences between natural images and biomedical images. Specifically, from a data-centric perspective, there are three sub-challenges. The first is the distinctive characteristics of biomedical images (e.g., intricate morphology and organization at multi-scales, from molecular to organ levels). We propose adaptive model learning approaches to enable the model to capture biomedical image characteristics more effectively. Such approaches are demonstrated by two studies: 1) NaviAirway, a tailored airway segmentation pipeline designed for the complex tree-like structure of airways in Computed Tomography (CT) scans and 2) 3DCellSeg, a cell segmentation pipeline specifically addressing the challenge of clumped cells in 3D cell membrane microscopic images. Besides distinctive characteristics, high expensive labeling and limited data sources pose the second sub-challenge. We hereby develop AICellCounter, a CPU-based tool that combines deep learning and traditional machine learning for counting target cells in confocal microscopy images. It only requires a single labeled sample for training and provides results within minutes. Lastly, we tackle the challenge of combining medical image datasets with different distributions by leveraging multimodal biomedical data. Our preliminary study utilizes prompt embeddings for model conditioning to enhance the performance of each individual set. In parallel, enhancing model interpretability is vital for high-stakes applications, especially for biomedical image analysis. The lack of explanations of a model's inner reasoning process can lead to serious ethical or legal problems when errors occur. To interpret models, we ask ``What concepts has a model learned, and which components of the model account for these concepts?'' We first devise a traditional machine learning model to explain the efficiency of Singular Value Decomposition (SVD) for clutter filtering in high frame-rate ultrasound imaging by finding which regions in an ultrasound image are associated with singular values. We then investigate deep networks to interpret the concepts learned by hidden layer neurons. We propose a HIerarchical Neuron concepT explainer (HINT) method, which systematically and quantitatively explains whether and how the neurons learn the high-level hierarchical relationships of concepts implicitly. We demonstrate the method's versatility across various tasks, such as evaluation of medical image classification models. In addition to associating neurons with concepts, we present another method named Spatial Activation Concept Vector (SACV) to quantify the contribution of a given concept to a model's prediction while overcoming the influence of redundant background features. To summarize, this thesis serves as a foundation towards effective and interpretable machine learning models for biomedical image analysis. This ultimate goal will be achieved with more in-depth model interpretations and more data-centric approaches for biomedical images.