Higher order speeded cognitive abilities depend on efficient coordination of activity across the brain, rendering them vulnerable to age reductions in structural and functional brain connectivity. The concept of “disconnected aging” has been invoked, suggesting that degeneration of connections between distant brain regions cause cognitive reductions. However, it has not been shown that changes in cognitive functions over time can be explained by simultaneous changes in brain connectivity. We followed 119 young and middle-aged (23-52 years) and older (63-86 years) adults for 3.3 years with repeated assessments of structural and functional brain connectivity and executive functions. We found unique age-related longitudinal reductions in executive function over and above changes in more basic cognitive processes. Intriguingly, 82.5% of the age-related decline in executive function could be explained by changes in connectivity over time. While both structural and functional connectivity changes were related to longitudinal reductions in executive function, only structural connectivity change could explain the age-specific decline. This suggests that the major part of the age-related reductions in executive function can be attributed to micro- and macrostructural alterations in brain connectivity. Although correlational in nature, we believe the present results constitute evidence for a “disconnected brain” view on cognitive aging.