Background: Echocardiography is essential to modern cardiology. However, human interpretation limits accurate and standardized high-throughput analysis, limiting echocardiography from reaching its full clinical and research potential for precision medicine. Deep learning has been useful in analyzing medical images but has not yet been widely applied to echocardiography, partly due to the complexity of its multi-view, multi-modality format. The essential first step toward comprehensive computer-assisted echocardiographic interpretation is determining whether computers can learn to recognize standard views of the heart.
Objectives: To test whether deep learning can accurately classify echocardiogram views.
Methods: We anonymized 834,267 transthoracic echocardiogram (TTE) images from 267 patients (20-96 years, 51percent female, 26 percent obese) seen between 2000 and 2017 and labeled these images according to standard views. Images covered a range of real-world clinical variation and acquisition. We built a multilayer convolutional neural network and used supervised learning to simultaneously classify 15 standard views. Eighty percent of data used was randomly chosen for training and 20 percent reserved for validation and testing on never-seen echocardiograms.
Results: Using multiple images from each clip, the model classified among 12 video views with 97.8% overall test accuracy without overfitting. Even on single low-resolution images, test accuracy among 15 views was 91.7 percent versus 70.2-83.5 percent for board-certified echocardiographers performing the same task. Confusional matrices, occlusion experiments, and saliency mapping showed that the model finds recognizable similarities among related views and classifies using clinically relevant image features.
Conclusions: Deep neural networks can classify essential echocardiographic views simultaneously and with high accuracy. Our results provide a foundation for more complex deep-learning-assisted echocardiographic interpretation.