Objective: Childhood cognitive ability is associated with lifestyle in adulthood, including self-reported physical activity (PA). We examined whether childhood cognitive ability is associated with objectively measured PA and sedentary time (ST) in young adulthood. Method: Participants of the Arvo Ylppö Longitudinal Study (n = 500) underwent tests of general reasoning, visuomotor integration, verbal competence, and language comprehension at the age of 56 months yielding a general intelligence factor score; at the age of 25 years they wore omnidirectional accelerometers for 9 days (range = 4–10 days) measuring overall daily PA (counts per minute, cpm), ST, and light and moderate-to-vigorous PA (MVPA) (minutes), and completed a questionnaire on occupational, commuting, leisure-time conditioning and nonconditioning PA. Results: After adjustment for sex, age, BMI-for-age SD score at 56 months. and mean of valid minutes of measurement period for PA, per each 1 SD increase in the childhood general intelligence factor score, overall daily PA decreased by −8.99 cpm/day, ST increased by 14.93 min/day, time spent in light PA decreased by −14.39 min/day, and the odds per each level increase in physical demandingness of the work and in time spent in nonconditioning leisure-time PA decreased by 38% and 31%, respectively (p values < 0.04). These associations were mediated via higher young adulthood level of education. Conclusions: In contrast to expected, in this cohort of young adults with high variability in PA, of whom many were still studying, higher childhood cognitive ability was associated with more objectively measured and self-reported physical inactivity. Whether these findings persist beyond young adulthood is a subject of further studies.