While there is consistent evidence that initial levels of cognitive ability predict mortality, there is mixed evidence for a relationship between changes in cognition and mortality. There have been few studies that have examined whether the level and slope of cognitive performance is predictive of subsequent mortality from all causes or from cardiovascular disease, stroke, heart disease, respiratory disease, or cancer. This study aimed to assess whether the level and slope of cognitive ability were associated with all-cause or cause-specific mortality. A cohort of 896 community-based elderly people in Australia was interviewed four times over 12 years, with vital status followed for up to 17 years. Of these, 592 participants completed two or more interviews and were included in survival models of six mortality outcomes. Cognitive change in five domains of ability was estimated using latent growth models. Poorer initial processing speed or verbal fluency was significantly associated with greater all-cause and/or cardiovascular mortality. In addition, declines in global ability were associated with greater all-cause, cardiovascular, and heart disease mortality. Vocabulary and episodic memory were not associated with mortality, and none of the cognitive tests significantly predicted respiratory or cancer mortality. Initial levels of cognitive ability tended to be better predictors of subsequent mortality than were changes in ability. The results suggest that vascular events may be largely responsible for the overall relationship between cognition and mortality.