The study introduces a math anxiety scale that systematically addresses psychological components, including cognitive (worry) and affective (nervousness) math anxiety when dealing with mathematical problems in mathematics-related settings (concerning tests, teachers, learning in class, working with mathematics textbooks, mathematics homework, and applying mathematics in everyday life). Our results indicate a hierarchical structure of math anxiety. Specifically, cognitive and affective math anxiety at the second-order level each determined three setting factors at the first-order level concerning evaluation (tests, teachers), learning (in class, with mathematics books, and during homework), and application (applying mathematics in everyday life). Furthermore, girls reported higher math anxiety than boys, which was particularly pronounced in the affective scale and in high-stakes academic settings, such as those involving evaluation and learning. After controlling for mathematics performance, gender effects decreased in all sub-dimensions but remained significant in affective math evaluation anxiety. Practical implications and directions for further research on cognitive and affective math anxiety are discussed.