Insulin-like growth factor 1 (IGF-1) has been associated with osteoporosis, cardiovascular disease, cancer, neurodegenerative diseases, and mortality in middle and older aged adults. Cross-sectionally, IGF-1 decreases with age and levels of IGF-1 are markedly different between individuals. However, little is known about intra-individual trajectories of IGF-1. We examined baseline and serial measures of plasma total IGF-1, IGF binding protein (IGFBP)-3, and their ratio, which is a proxy for bioavailable IGF-1, among 1618 adults, aged 50–95, enrolled in the Mayo Clinic Study of Aging. At baseline, IGF-1 and IGFBP-3 were strongly correlated (r = 0.62, p < 0.001). Total IGF-1 and IGFBP-3 decreased across age, while the ratio of IGF-1/IGFBP-3 increased across age. This pattern was consistent across ages at baseline and intra-individually over an average 2.3 years follow-up (range = 10 months–5.6 years). In age-adjusted linear regression models, baseline levels of total IGF-1, IGFBP-3, and IGF-1/IGFBP-3 varied by participant characteristics (sex, BMI, gait speed), medical comorbidities (Charlson comorbidity index score, hypertension, diabetes, and cardiovascular disease), and hormone replacement therapy use in women. High interclass correlation coefficients (ICCs) suggest little intra-individual variability in levels of total IGF-1 (ICC = 0.84), IGFBP-3 (ICC = 0.88), and IGF-1/IGFBP-3 (ICC = 0.81) over time. In mixed effects models that specified age as a time scale, men showed greater decreases in total IGF-1 and IGFBP-3 with age, while more comorbidities and decreasing gait speed were associated with increasing IGFBP-3. In sex-stratified models, trajectories of total IGF-1, IGFBP-3, and IGF-1/IGFBP-3, as a function of participant demographics, health characteristics, and medical conditions, differed between men and women. These results suggest that change in levels of plasma total IGF-1, IGFBP-3, and IGF-1/IGFBP-3 are associated with demographics, health characteristics, and medical conditions, and that the trajectories of change differ by sex. Future research should consider how IGF-1 and IGFBP-3 might be useful in research or clinic, paying particular attention to how sex may impact levels as a function of demographics, health characteristics, and medical conditions.