We lack a mechanistic explanation for the stereotyped pattern of white matter loss seen in Huntington’s disease. While the earliest white matter changes are seen around the striatum, within the corpus callosum and in the posterior white matter tracts, the order in which these changes occur and why these white matter connexions are specifically vulnerable is unclear. Here we use diffusion tractography in a longitudinal cohort of individuals yet to develop HD to identify a hierarchy of white matter connexion vulnerability, where cortico-striatal connexions are most affected, followed by inter-hemispheric and intra-hemispheric connexions. We provide a mechanistic explanation for this hierarchy by showing that the physical length of white matter connexions between a brain area and its neighbours predicts the rate of atrophy over 24 months. This finding demonstrates a new principle underlying neurodegeneration in Huntington’s disease, whereby the longest brain connexions are the first to suffer damage that can account for the stereotyped pattern of white matter loss observed in premanifest Huntington’s disease.