A novel and fully automated system of Cobb angle detection using deep learning: improving diagnosis and monitoring of adolescent idiopathic scoliosis

Dr Cheung, Jason Pui Yin   (Principal Investigator (PI))

Dr Zhang Teng Grace   (Co-Investigator)

36

2020-06-15

150000

A novel and fully automated system of Cobb angle detection using deep learning: improving diagnosis and monitoring of adolescent idiopathic scoliosis

Adolescent idiopathic scoliosis, automation, Cobb angle, Deep learning

Orthopaedics/TraumatologyArtificial Intelligence and Machine learning

201910160007

Seed Fund for PI Research – Translational and Applied Research

2019

On-going

Adolescent idiopathic scoliosis (AIS) is a 3-dimensional deformity commonly characterized as a ""S"" or ""C"" shaped lateral curvature.[6] It is the most common spine condition in adolescents with up to 2.2% of boys and 4.8% of girls have scoliosis in Hong Kong.[9] Without prompt intervention, this spinal deformity may progress and significantly reduce the quality of life and mobility of these children. In addition to the physical, emotional and psychological strain to these children, there is an increased economic burden on families and healthcare system. Diagnosis of AIS is reliant on the clinician’s knowledge and experience. The current assessment tools for spinal alignment include physical examinations and medical imaging.[4] For physical examination, the patient is assessment in standing posture to identify any back asymmetry. However, this is highly subjective and is often not easy for the human eye to discern. Back appearance may not be easy to identify any curve progression or poor body posture. Hence, imaging assessment is necessary with x-rays.[7] While this is objective, the interpretation of imaging data is relatively subjective and therefore can contribute to inter-rater variance. For treatment planning, imaging is usually used to determine ideal alignment parameters[8, 11, 17, 25], fusion level selection, and designing corrective and instrumentation strategies.[5, 12, 14, 18, 22, 28] X-rays provide information of curve magnitude[4] via the Cobb angle measurement and balance/alignment parameters. Repeated X-rays are required at nearly every routine follow-up to have close monitoring of the curve progression and to guide treatment. Despite newer advances like the biplanar stereoradiography, repeated X-rays are associated with a large radiation exposure which may potentially increase the risk of cancer especially in young growing children. Patients and families also carry anxiety at every consultation and between these visits waiting for results of their x-rays without real-time feedback on the progress of the disease. There is a lack of an easily accessible and objective tool to objectively assess the spinal deformity severity and progression. Such a tool is crucial for assisting clinicians in monitoring treatment outcomes and improving the patient’s compliance with treatment by evidence of improvement with longitudinal treatment. The field of computer vision with artificial intelligence (AI) technology is applicable to the current diagnostic challenges of AIS by providing automated, objective and consistent feature detection of images using machine learning. There are previous research works on Cobb angle computation from X-ray images, which often first detect the locations of vertebrae and then compute Cobb angles based on the detected vertebrae.[10] Traditional methods rely on digital image processing techniques to segment the vertebrae.[1-3, 16, 21, 24] For example, Non-Local Means for image denoising and Canny edge detection were performed on a manually selected region of interest to obtain vertebra slopes.[16] The Cobb angle is then computed from the slopes of the end vertebrae. Due to the large anatomical variance of the spine and low image quality of X-rays (e.g., poor contrast/resolution), these handcrafted methods often have limited accuracy, and not applicable for clinical use. Inspired by the great success of deep learning-based methods in various computer vision problems, convolutional neural networks (CNNs) have been exploited by researchers in vertebral landmark detection and Cobb angle computation.[13, 30-32] Compared with traditional handcrafted methods, deep learning-based methods can be trained end-to-end via a data-driven manner and can potentially better handle different challenges encountered in working automated Cobb angle computation. A semi-automated algorithm for Cobb angle computation was introduced.[32] Users are required to manually select several patches containing the upper and lower end vertebrae that are used to directly regress the Cobb angle. Furthermore, a CNN was used previously to directly regress the pixel coordinates of the vertebral landmarks.[30] Multi-view X-ray images (i.e., frontal and lateral views) for vertebral landmark detection and Cobb angle regression was proposed previously as well.[31] Spine segmentation using a U-Net[23] like network were done[13] and Cobb angle was computed directly based on the segmentation results and the Cobb angle criterion. There are also methods that directly regress Cobb angle from input images.[26] A common shortcoming of these learning-based methods is that they were all trained and tested only on a very small dataset. The size of the previous dataset was 275 images[32], 481 images[30], 526 images[31], 595 images[13]. Given the limited size of the dataset, they are unlikely to generalize well to new data. Methods that compute Cobb angles from detected vertebrae by finding maximum angles formed by the endplates may produce results that do not conform to a feasible spinal deformity. For methods that directly regress Cobb angle from images or vertebral landmarks, it is impossible to guarantee the model learns the correct computation of Cobb angle without any intermediate supervision. Therefore, we propose to develop a fully automated system of Cobb angle detection, to reduce the laborious alignment measurements tasks as well as increasing the accuracy of measurements. Importantly, for deep learning purpose, a big dataset is required, and our unit has the advantage in a big database resource. Due to our long-standing school screening program since 1995, we have over 3000 patients with AIS seen in our unit with stored images per year. As of date, the screening program has screened over 1,000,000 children. The existing infrastructure provides a great platform for patient recruitment. Hence, we have extensive experience in the prospective collection of data from patients with spinal disorders including clinical and radiographic information. We aim to improve the current clinical care for AIS patients, by providing a novel and fully automated system of analyzing x-rays using deep learning to improve diagnosis and monitoring of patients with AIS. This can aid the clinicians for improved patient care, by potentially design effective treatment plans and adjust the treatment plans based on individual response to the previous treatment. Furthermore, the tool can potentially assist patient follow-ups or patient recruitment for future clinical research. Objective 1: To develop a deep convolutional neural network (CNN) for vertebral landmark detection on X-ray images of the spine and carry out extensive experiments and ablation study to evaluate the performance of the proposed network. Objective 2: To develop a deep convolutional neural network (CNN) for end vertebra detection on X-ray images of the spine and carry out extensive experiments and ablation study to evaluate the performance of the proposed network and the accuracy of Cobb angles computed from the vertebral landmarks of the detected end vertebrae.