The detection and appreciation of humor is a complex cognitive process that remains poorly understood. Although functional neuroimaging studies have begun to map the brain systems involved in humor appreciation, there are virtually no data on the structural correlates between gray matter volume and this capacity. Using voxel-based morphometry, the present study examined the association between gray matter volume and the ability to detect and appreciate humor. Fifty-nine healthy adults aged 18–45 years (30 men) underwent structural MRI and completed the University of Pennsylvania Humor Appreciation Test (HAT). After controlling for age and sex, gray matter volume of the left inferior frontal gyrus, left temporal pole, and left insula correlated positively with the appreciation of visual and verbal humor on the HAT, whereas the gray matter volume of the right inferior frontal gyrus correlated only with verbal humor appreciation scores. There were no negative correlations between gray matter volume and HAT performance. These data support a neurobiological basis for humor appreciation, particularly involving left-hemispheric cortical systems, and further suggest that individual differences in humor appreciation may be related to differences in regional gray matter volume.