Parental ages are increasing in the developed world, and postponed parenthood may have a negative association with the cognitive ability of offspring. There is, however, inconclusive evidence regarding the impact of both maternal and paternal ages. We have been able to reduce or eliminate unobserved confounding by using methods that account for fixed parental characteristics shared by brothers. Associations between parental age and intelligence quotient (IQ) among 565,433 Swedish males (birth cohorts 1951 to 1976) were analyzed, with IQ measured at conscription examinations (given between ages 17 and 20 years). When we accounted for the IQ time trend by adjusting for birth year, advanced paternal age showed no association with offspring IQ; however, maternal ages above 30 years were inversely associated with offspring IQ. For example, maternal ages 40–44 years were associated with an offspring IQ that was 0.07 standard deviations lower than that for maternal ages 25–29 years (P < 0.001). However, the IQ trend more than offset the impact of age, as without birth year adjustment, advanced maternal age was positively associated with IQ. Although the results confirmed that maternal age was negatively associated with offspring IQ, the association was small enough that delaying parenthood resulted in higher offspring IQ scores because of the positive IQ test score trend.