Observations in witnessed Sudden Unexpected Death in Epilepsy (SUDEP) suggest that a fatal breakdown of the central autonomic control could play a major role in SUDEP. A previous MR study found volume losses in the mesencephalon in focal epilepsy that were more severe and extended into the lower brainstem in two patients who later died of SUDEP. The aims of this study were to demonstrate an association (1) between brainstem volume loss and impaired autonomic control (reduced heart rate variability [HRV]); (2) between brainstem damage and time to SUDEP in patients who later died of SUDEP. Two populations were studied: (1) Autonomic system function population (ASF, 18 patients with focal epilepsy, 11 controls) with HRV measurements and standardized 3 T MR exams. (2) SUDEP population (26 SUDEP epilepsy patients) with clinical MRI 1–10 years before SUDEP. Deformation-based morphometry of the brainstem was used to generate profile similarity maps from the resulting Jacobian determinant maps that were further characterized by graph analysis to identify regions with excessive expansion indicating significant volume loss or atrophy. The total number of regions with excessive expansion in ASF was negatively correlated with HRV (r = −.37, p = .03), excessive volume loss in periaqueductal gray/medulla oblongata autonomic nuclei explained most of the HRV associated variation (r/r2 = −.82/.67, p < .001). The total number of regions with excessive expansion in SUDEP was negatively correlated with time to SUDEP (r = −.39, p = .03), excessive volume loss in the raphe/medulla oblongata at the obex level explained most of the variation of the time between MRI to SUDEP (r/r2 = −.60/.35, p = .001). Epilepsy is associated with brainstem atrophy that impairs autonomic control and can increase the risk for SUDEP if it expands into the mesencephalon.