Physical capability in later life is influenced by factors occurring across the life course, yet exposures to area conditions have only been examined cross-sectionally. Data from the National Survey of Health and Development, a longitudinal study of a 1946 British birth cohort, were used to estimate associations of area deprivation (defined as percentage of employed people working in partly skilled or unskilled occupations) at ages 4, 26, and 53 years (residential addresses linked to census data in 1950, 1972, and 1999) with 3 measures of physical capability at age 53 years: grip strength, standing balance, and chair-rise time. Cross-classified multilevel models with individuals nested within areas at the 3 ages showed that models assessing a single time point underestimate total area contributions to physical capability. For balance and chair-rise performance, associations with area deprivation in midlife were robust to adjustment for individual socioeconomic position and prior area deprivation (mean change for a 1-standard-deviation increase: balance, −7.4% (95% confidence interval (CI): −12.8, −2.8); chair rise, 2.1% (95% CI: −0.1, 4.3)). In addition, area deprivation in childhood was related to balance after adjustment for childhood socioeconomic position (−5.1%, 95% CI: −8.7, −1.6). Interventions aimed at reducing midlife disparities in physical capability should target the socioeconomic environment of individuals—for standing balance, as early as childhood.