A wealth of neuroimaging research has associated semantic variant primary progressive aphasia with distributed cortical atrophy that is most prominent in the left anterior temporal cortex; however, there is little consensus regarding which region within the anterior temporal cortex is most prominently damaged, which may indicate the putative origin of neurodegeneration. In this study, we localized the most prominent and consistent region of atrophy in semantic variant primary progressive aphasia using cortical thickness analysis in two independent patient samples (n = 16 and 28, respectively) relative to age-matched controls (n = 30). Across both samples the point of maximal atrophy was located in the same region of the left temporal pole. This same region was the point of maximal atrophy in 100% of individual patients in both semantic variant primary progressive aphasia samples. Using resting state functional connectivity in healthy young adults (n = 89), we showed that the seed region derived from the semantic variant primary progressive aphasia analysis was strongly connected with a large-scale network that closely resembled the distributed atrophy pattern in semantic variant primary progressive aphasia. In both patient samples, the magnitude of atrophy within a brain region was predicted by that region’s strength of functional connectivity to the temporopolar seed region in healthy adults. These findings suggest that cortical atrophy in semantic variant primary progressive aphasia may follow connectional pathways within a large-scale network that converges on the temporal pole.