Comprehensive utilization of medicinal plant resources is of great significance for sustainable development of traditional Chinese medicines. In the present study, the α-glucosidase inhibitory activities of the rhizome and fibrous root of Anemarrhena Asphodeloides Bunge, were compared detailedly, and a high performance liquid chromatography coupled with electrospray ionization tandem triple quadrupole mass spectrometry (HPLC-QQQ/MS) method was developed for simultaneous quantification of seven bioactive constituents including neomangiferin, mangiferin, isomangiferin, timosaponin BII, timosaponin B, timosaponin AIII, and timosaponin N in 40 batches of samples. The results demonstrated that fibrous root extracts had more potent α-glucosidase inhibitory activity than rhizome extracts. Mangiferin and isomangiferin were abundant in fibrous root, while the analyzed saponins were rich in rhizome. Based on the chemometrics methods including principal component analysis (PCA), orthogonal partial least square discriminant analysis (OPLS-DA), and partial least square (PLS), mangiferin and isomangiferin might be mainly responsible for α-glucosidase inhibitory activity of the genus. These findings indicate that the established HPLC-QQQ/MS method was proven to be useful and efficient for quality control of Anemarrhena materials, and fibrous root had the potential to be utilized as anti-diabetic medicinal resource.