The explanation for why significant sex differences are found in creativity has become an increasingly important topic. The current study applied a cognitive neuroscience perspective and voxel-based morphometry to investigate the sex differences for the association between verbal creativity and gray matter volume (GMV) in a large sample of healthy adults from the Chinese Mainland (163 men and 193 women). Furthermore, we sought to determine which brain regions are responsible for these differences. Our behavioral results showed a significant sex difference. Specifically, women scored higher than men on originality. The voxel-based morphometry results indicated that the relationship between originality and GMV differed between men and women in the left temporo-occipital junction. Higher originality scores in women were associated with more GMV. In contrast, higher originality scores in men were related to less GMV. These findings suggest the left temporo-occipital junction GMV plays a unique role in the sex differences in verbal creativity because women usually surpass men in semantic processing, which is the major function of the left temporal region.