Stenosis-DetNet: Sequence consistency-based stenosis detection for X-ray coronary angiography
Introduction
Coronary stenosis automatic detection on X-ray contrast images is important in coronary heart disease diagnosis. In X-ray images, the stenosis is usually shown in different positions along the artery; the most serious of which is that at the junction of the main artery. The vascular features on single-frame angiographic images are weakened by stenosis, existence of thrombus, imaging conditions, and other factors. Vascular overlap, pseudo-bifurcation, uneven gray level of vascular development, and other problems may also occur. In addition, the coronary artery is in a state of non-stop movement in imaging because of heartbeat and respiratory movement, which cause angiographic image degradation and motion artifacts and seriously affect stenosis diagnosis. The identification of stenosis (Janssen et al., 2010) is subject to the doctor’s judgment in clinic. Therefore, computer-aided diagnosis and treatment are necessary to detect coronary artery stenosis. Such systems can eliminate the differences between different observers, provide doctors with an objective reference, and improve the efficiency of diagnosis and treatment.
The detection of coronary artery stenosis is the most important step in coronary disease evaluation methods, such as Leaman (Leaman et al., 1981), Gensini (Gensini, 1983), and SYNTAX (Sianos et al., 2005) score. In clinical practice, these evaluation methods are used to quantitatively analyze the scope and severity of the lesions, and to provide a basis for the selection of appropriate clinical treatment. In surgery, real-time stenosis detection is important to remind doctors to focuse on goals. Thus, the reliable detection of coronary artery stenosis is significant for preoperative diagnosis and intraoperative guidance.
The rapid reduction of vessel diameter is the most direct indication of stenosis on X-ray contrast images. Brieva et al. (Brieva et al., 2004) used deformable splines to extract the edges of blood vessels, calculated the diameter of blood vessels, and provided the results of stenosis detection. Compas et al. (Compas et al., 2014) calculated the vascular diameter based on the gray change of the image and combined the vascular tracking results of sequence images to generate the vascular diameter curved surface. The minimum value of the curved surface corresponds to vascular stenosis. Mohan et al. (Mohan and Vishnukumar, 2017) calculated the diameter of vessels after vascular enhancement by using a morphological method; the area where the diameter suddenly decreased was labeled as stenosis. Wan et al. (Wan et al., 2018) used the level set skeletonization method to extract the vascular tree structure and accurately calculated the vascular diameter and direction on the basis of the improved matching measure model. The location and degree of vascular stenosis were divided by calculating the ratio of the minimum point to the maximum point of the diameter curve. The results of coronary artery segmentation are often used to calculate the vessel diameter. The radius of vessels is calculated on the basis of coronary artery segmentation to detect or grade stenosis (Au et al., 2018; M’hiri et al., 2017; Nandhu Kishore and Jayanthi, 2019). Stenosis detection based on vascular diameter analysis requires a highly accurate of vascular tree structure extraction. Thus, ensuring accurate stenosis detection is difficult.
X-ray images can be directly processed using various methods to exclude the influence of vessel segmentation and vessel centerline extraction on coronary stenosis detection accuracy. Du et al. (Du et al., 2018) designed a multi-scale convolution neural network to extract the texture features at different scales in X-ray images and then used the Faster-RCNN framework (Ren et al., 2015) to detect and locate the stenosis. Cong et al. (Cong et al., 2019) used a convolutional neural network and recurrent neural network model to select keyframes and classify coronary artery stenosis. Stenosis detection methods based on vascular characteristics are more effective than those based on vascular diameter. However, a serious imbalance exists between the positive and negative samples in the training data, because the area of vascular stenosis accounts for a small proportion of the whole image. Therefore, the accuracy of stenosis detection results is insufficient to fulfill the requirements of doctors when making a clinical diagnosis.
Doctors and experts often use the sequence information of angiographic images in the manual diagnosis of coronary artery stenosis. Combined with temporal information, the accuracy of many methods for coronary artery stenosis detection can be improved. Zhang et al. (Zhang et al., 2019) analyzed the degree of coronary artery stenosis by using two 3D convolution neural networks (CNNs) combined with attention mechanism to extract the features of X-ray coronary angiographic image sequences from two different perspectives and then used a 2D residual network to extract the features of a keyframe image. Finally, the three features were fused to determine coronary artery stenosis degree. This method can only identify the stenosis on the branches of vessels. It cannot identify stenoses at the junction of different branches, such as bifurcation and trigeminal stenosis. Wu et al. (Wu et al., 2020) used advanced object detection framework to accurately detect coronary artery stenosis. They did not use the sequence image information in the detection process. Only the information to constrain the detection results was used. Hence, the stenosis detection results were not robust.
In this study, we proposed Stenosis-DetNet based on sequence information to accurately locate coronary artery stenosis in X-ray coronary angiographic images by maximizing the use of sequence information. The method extracts the candidate object’s bounding box and feature map from the continuous frames of the sequence contrast images. Then, sequence feature fusion module (SFF) is performed on the feature maps of all candidate objects in the continuous frames. Classification and regression are used to obtain the initial detection results of each frame image on the new feature map after feature fusion. Finally, the coronary artery displacement information and coronary artery feature information in the sequence image are used for accurate stenosis detection. The main contributions of this paper are as follows. 1) A coronary artery stenosis detection network is designed to maximize the sequence image information and achieve fast and accurate stenosis detection. 2) A feature fusion module is added to fuse the continuous images information in the sequence, so that the network can learn more features of stenosis. 3) The characteristic of small displacement of the vascular in sequence images is used to optimize the stenosis detection results of the network, which effectively improves the performance of the algorithm.
Section snippets
Datasets
A total of 166 X-ray sequence angiographic data were collected from Peking Union Medical College Hospital and Anzhen Hospital. And these were used to verify the effectiveness of Stenosis-DetNet in detecting vascular stenosis on sequential angiographic images. The image resolution is 512 × 512, and the frame rate is 15FPS. In the experiment, 9 consecutive angiographic images with complete vascular filling and clear imaging were selected from each sequence. The target bounding boxes of coronary
Experimental design and evaluation metrics
The experiments designed in this paper include the analysis of the performance of the Stenosis-DetNet module, the comparison of the performance of the algorithm with other object detection methods, and the comparison between the algorithm and other stenosis detection methods. The stenosis detection effect of the sequence feature fusion module and sequence consistency alignment module are be displayed and analyzed in the performance analysis of the algorithm module. In the part of algorithm
How to avoid the overfitting issues
To prevent the over-fitting problem, we have adopted three strategies, as follows:
Dataset enhancement: There were 140 XRA sequence data in training set, and each sequence contained 9 clear angiographic images with a total of 1260 images. In the process of training, we amplified the dataset, that is, translated, flipped, zoomed and performed other operations on all images. The size of the dataset doubled to 2520 images.
Transfer learning: In the process of training, the feature extraction module
Conclusion
We propose a coronary artery stenosis detection network based on sequence image information. This method maximizes the sequence image information and proposes a sequence feature fusion module and a sequence consistency alignment module. It can suppress false-positive and false-negative results of stenosis detection in sequence angiographic images.
Experiments are carried out for the proposed sequence feature fusion module and sequence consistency alignment module, and the method is compared with
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
This work was supported by the National Key R&D Program of China [2019YFC0119300], and the National Science Foundation Program of China (62071048, 62025104, 61971040, 61901031).
References (26)
A more meaningful scoring system for determining the severity of coronary heart disease
Am. J. Cardiol.
(1983)- et al.
Automated identification and grading of coronary artery stenoses with X-ray angiography
Comput. Methods Programs Biomed.
(2018) - et al.
Automatic detection of coronary artery stenosis by convolutional neural network with temporal constraint
Comput. Biol. Med.
(2020) - et al.
Automated Characterization of Stenosis in Invasive Coronary Angiography Images With Convolutional Neural Networks
(2018) - et al.
Coronary extraction and stenosis quantification in X-Ray angiographic imaging
- et al.
Cascade R-CNN: delving into high quality object detection
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(2018) - et al.
MMDetection: Open MMLab Detection Toolbox and Benchmark
(2019) - et al.
Automatic detection of coronary stenosis in X-ray angiography through spatio-temporal tracking
- et al.
Automated stenosis detection and classification in X-ray angiography using deep neural network
- et al.
Real-time lesion detection of cardiac coronary artery using deep neural networks