Lifetime exposures to adverse social environments influence adult health, as do exposures in early life. It is usual to examine the influences of school on teenage health and of adult area of residence on adult health. We examined the combined long-term association of the school attended, as well as the area of residence in childhood, with adult health. A total of 6,285 children from Aberdeen, Scotland, who were aged 5–12 years in 1962, were followed up at a mean age of 47 years in 2001. Cross-classified multilevel logistic regression was used to estimate the associations of family, school, and area of residence with self-reported adult health and mental health, adjusting for childhood family-, school-, and neighborhood-level factors, as well as current adult occupational position. Low early-life social position (as determined by the father's occupational level) was associated with poor adult self-rated health but not poor mental health. There were small contextual associations between childhood school environment (median odds ratio = 1.08) and neighborhood environment (median odds ratio = 1.05) and adult self-rated health. The share of the total variance in health at the family level was 10.1% compared with 89.6% at the individual level. Both socioeconomic context and composition in early life appear to have an influence on adult health, even after adjustment for current occupational position.