We examined relationships between regional brain shrinkage and changes in cognitive performance, while taking into account the influence of chronological age, vascular risk, Apolipoprotein E variant and socioeconomic status. Regional brain volumes and cognitive performance were assessed in 167 healthy adults (age 19–79 at baseline), 90 of whom returned for the follow-up after two years. Brain volumes were measured in six regions of interest (ROIs): lateral prefrontal cortex (LPFC), prefrontal white matter (PFw), hippocampus (Hc), parahippocampal gyrus (PhG), cerebellar hemispheres (CbH), and primary visual cortex (VC), and cognitive performance was evaluated in three domains: episodic memory (EM), fluid intelligence (Gf), and vocabulary (V). Average volume loss was observed in Hc, PhG and CbH, but reliable individual differences were noted in all examined ROIs. Average positive change was observed in EM and V performance but not in Gf scores, yet only the last evidenced individual differences in change. We observed reciprocal influences among neuroanatomical and cognitive variables. Larger brain volumes at baseline predicted greater individual gains in Gf, but differences in LPFC volume change were in part explained by baseline level of cognitive performance. In one region (PFw), individual change in volume was coupled with change in Gf. Larger initial brain volumes did not predict slower shrinkage. The results underscore the complex role of brain maintenance and cognitive reserve in adult development.