Ilex rotunda, in which triterpenoids and phenylpropanoids are major bioactive constituents, has been widely used in traditional Chinese medicines. In this study, a validated UPLC/Q-TOF-MS/MS method was developed to simultaneously identify and quantify the triterpenoids and phenylpropanoids in the stem bark, fruit, leaves, roots and stem xylem of this herbal medicine. A total of seventy triterpenoids and twelve phenylpropanoids were identified with the assistance of the modified mass defect filter and key product ion filter data processing strategies, and forty-eight of them were confirmed by reference substances. Meanwhile, the contents of twelve triterpenoids and three phenylpropanoids in the five plant parts were determined with good linearity (R2≥0.9993), precision (RSD≤2.04%), repeatability (RSD≤1.99%), stability (RSD≤1.88%) and recovery (96.65–103.17% and RSD≤3.54%). Furthermore, PCA and OPLS-DA methods were employed to visualize the relationships and discrimination of the forty-two stem bark samples from two origins based on the contents of fifteen analytes. Our findings may provide early scientific evidence for quality control and for elucidating the therapeutic principle of Ilex rotunda.