Cerebral hypoperfusion elevates the risk of brain white matter (WM) lesions and cognitive impairment. Central artery stiffness impairs baroreflex, which controls systemic arterial perfusion, and may deteriorate neuronal fiber integrity of brain WM. The purpose of this study was to examine the associations among brain WM neuronal fiber integrity, baroreflex sensitivity (BRS), and central artery stiffness in older adults. Fifty-four adults (65 ± 6 years) with normal cognitive function or mild cognitive impairment (MCI) were tested. The neuronal fiber integrity of brain WM was assessed from diffusion metrics acquired by diffusion tensor imaging. BRS was measured in response to acute changes in blood pressure induced by bolus injections of vasoactive drugs. Central artery stiffness was measured by carotid–femoral pulse wave velocity (cfPWV). The WM diffusion metrics including fractional anisotropy (FA) and radial (RD) and axial (AD) diffusivities, BRS, and cfPWV were not different between the control and MCI groups. Thus, the data from both groups were combined for subsequent analyses. Across WM, fiber tracts with decreased FA and increased RD were associated with lower BRS and higher cfPWV, with many of the areas presenting spatial overlap. In particular, the BRS assessed during hypotension was strongly correlated with FA and RD when compared with hypertension. Executive function performance was associated with FA and RD in the areas that correlated with cfPWV and BRS. These findings suggest that baroreflex-mediated control of systemic arterial perfusion, especially during hypotension, may play a crucial role in maintaining neuronal fiber integrity of brain WM in older adults.