The shear wave elastography (SWE) uses the acoustic radiation force to measure the stiffness of tissues and is less operator dependent in data acquisition compared to strain elastography. However, the reproducibility of the result is still interpreter dependent. The purpose of this study is to develop a computer-aided diagnosis (CAD) method to differentiate benign from malignant breast tumors using SWE images. After applying the level set method to automatically segment the tumor contour and hue-saturation-value color transformation, SWE features including average tissue elasticity, sectional stiffness ratio, and normalized minimum distance for grouped stiffer pixels are calculated. Finally, the performance of CAD based on SWE features are compared with those based on B-mode ultrasound (morphologic and textural) features, and a combination of both feature sets to differentiate benign from malignant tumors. In this study, we use 109 biopsy-proved breast tumors composed of 57 benign and 52 malignant cases. The experimental results show that the sensitivity, specificity, accuracy and the area under the receiver operating characteristic ROC curve (Az value) of CAD are 86.5%, 93.0%, 89.9%, and 0.905 for SWE features whereas they are 86.5%, 80.7%, 83.5% and 0.893 for B-mode features and 90.4%, 94.7%, 92.3% and 0.961 for the combined features. The Az value of combined feature set is significantly higher compared to the B-mode and SWE feature sets (p = 0.0296 and p = 0.0204, respectively). Our results suggest that the CAD based on SWE features has the potential to improve the performance of classifying breast tumors with US.