Microsatellite variants in the arginine vasopressin receptor 1A gene (AVPR1A) RS3 have been associated with normal social behaviors variation and autism spectrum disorders (ASDs) in a sex-specific manner. However, neural mechanisms underlying these associations remain largely unknown. We hypothesized that AVPR1A RS3 variants affect altruistic behavior by modulating the gray matter volume (GMV) of specific brain regions in a sex-specific manner. We investigated 278 young healthy adults using the Dictator Game to assess altruistic behavior. All subjects were genotyped and main effect of AVPR1A RS3 repeat polymorphisms and interaction of genotype-by-sex on the GMV were assessed in a voxel-wise manner. We observed that male subjects with relatively short repeats allocated less money to others and exhibited a significantly smaller GMV in the right fusiform face area (FFA) compared with male long homozygotes. In male subjects, the GMV of the right FFA exhibited a significant positive correlation with altruistic behavior. A mixed mediation and moderation analysis further revealed both a significant mediation effect of the GMV of the right FFA on the association between AVPR1A RS3 repeat polymorphisms and allocation sums and a significant moderation effect of sex (only in males) on the mediation effect. Post hoc analysis showed that the GMV of the right FFA was significantly smaller in male subjects carrying allele 426 than in non-426 carriers. These results suggest that the GMV of the right FFA may be a potential mediator whereby the genetic variants in AVPR1A RS3 affect altruistic behavior in healthy male subjects. Hum Brain Mapp 37:2700–2709, 2016. © 2016 Wiley Periodicals, Inc.