Both cognitive aging and β-amyloid (Aβ) deposition, a pathological hallmark of Alzheimer's disease, are associated with structural and cognitive changes in cognitively normal older people. To examine independent effects of age and Aβ deposition on cognition and brain structure in aging, 83 cognitively normal older adults underwent structural magnetic resonance imaging scans and neuropsychological tests and were classified as negative (PIB−) or positive (PIB+) for Aβ deposition using the radiotracer Pittsburgh compound B (PIB). Weighted composite discriminant scores represented subjects' cognition. Older adults showed age-related gray matter (GM) atrophy across the whole brain regardless of Aβ deposition. Amyloid burden within PIB+ subjects, however, was associated with GM atrophy in the frontal, parietal, and temporal cortices. Associations between cognition and volume in PIB− subjects were primarily seen throughout frontal regions and the striatum, while, in PIB+ subjects, these associations were seen in orbital-frontal and hippocampal regions. Furthermore, in PIB− subjects, cognition was related to putaminal volume, but not to hippocampus, while, in PIB+ subjects, cognition was related to hippocampal volume, but not to putamen. These findings highlight differential age and Aβ effects on brain structure, indicating effects of age and Aβ that operate somewhat independently to affect frontostriatal and medial temporal brain systems.