The single-nucleotide polymorphism rs9804190 in the Ankyrin G (ANK3) gene has been reported in genome-wide association studies to be associated with bipolar disorder (BD). However, the neural system effects of rs9804190 in BD are not known. We investigated associations between rs9804190 and gray and white matter (GM and WM, respectively) structure within a frontotemporal neural system implicated in BD. A total of 187 adolescent and adult European Americans were studied: a group homozygous for the C allele (52 individuals with BD and 56 controls) and a T-carrier group, carrying the high-risk T allele (38 BD and 41 controls). Subjects participated in high-resolution structural magnetic resonance imaging and diffusion tensor imaging (DTI) scanning. Frontotemporal region of interest (ROI) and whole-brain exploratory analyses were conducted. DTI ROI-based analysis revealed a significant diagnosis by genotype interaction within the uncinate fasciculus (P ≤ 0.05), with BD subjects carrying the T (risk) allele showing decreased fractional anisotropy compared with other subgroups, independent of age. Genotype effects were not observed in frontotemporal GM volume. These findings support effects of rs9804190 on frontotemporal WM in adolescents and adults with BD and suggest a mechanism contributing to WM pathology in BD.