A rapid, green, low cost and nondestructive attenuated total reflection near infrared (ATR NIR) method was developed to quantify the total polysaccharide and the main monosaccharides mannose and glucose in Dendrobium huoshanense. Total 100 D. huoshanense samples from different places were analyzed using ATR NIR method. Potential outlying samples were initially removed from the collected NIR data using the PCA-Mahalanobis distance method. Spectral data preprocessing was studied in the construction of a partial least squares (PLS) model and six different signal pretreatment methods, including multiplicative scattering correction (MSC), standard normal transformation (SNV), first and second derivatives, the combination of MSC with the first derivative, and the combination of SNV with the first derivative, were compared. The results showed that the best signal pretreatment method was the spectral data pretreated by SNV combined with the first derivative due to it showed the lowest root-mean-square error of cross-validation (RMSECV), highest R2 for both the polysaccharide and its main monosaccharides. In order to improve the performance of the model, the pretreated full spectrum was calculated by different wavelength selection method. The results showed that the optional wavelength selection model was the one simultaneously selecting the NIR wavelength ranges 7500–5750 cm−1, 5250–4700 cm−1, 4450–4300 cm−1 and 4200–4100 cm−1 because of the lowest RMSECV and the highest R2 among the ten wavelength selection models. The external validation and the complete external validation confirmed the robustness and reliability of the developed NIR model. The contents of the total polysaccharide and the main monosaccharides are the essential quality assessment criterion for plant medicines while their traditional quantification methods involved sample destruction, tedious sample processing and non-environmentally friendly pretreatment, therefore, our study might provide an efficient technique tool for the rapid, green and nondestructive quantification of the total polysaccharide and the main monosaccharides for D. huoshanense and other rich-in-polysaccharide plant medicines.