The structure and organization of cognitive abilities has been examined across the life span. The current analysis had three specific aims: (1) test the factor structure of a broad cognitive ability battery across three age groups; (2) examine differences in the pattern of factor covariation across age groups; and (3) examine the pattern of factor mean differences across age groups. A sample of 512 older African Americans (mean age=66.6 years, 25.4% male) from the Baltimore Study of Black Aging was administered a battery of cognitive tests assessing the domains of perceptual speed, verbal memory, inductive reasoning, vocabulary, and working memory. Factor models were estimated separately in middle-age adults (50–59 years, n=107), young-old adults (60–69 years, n=198), and old-old adults (70–79 years, n=207). There was loading invariance across the three age groups that suggests that the selected tests measured cognition similarly across age. There was no evidence of dedifferentiation across increasingly older age groups. Factor mean differences were observed with the middle-age group having significantly higher factor means than the young-old and old-old groups; however, there was only one factor mean difference between the young-old and the old-old groups. The results suggest that a pattern of dedifferentiation of cognitive abilities does not exist within this sample of older African Americans and that the 60–69 year age range may be a critical period for cognitive decline in this population.