Well-being and physical health are central indicators of quality of life in old age. Research from a between-person difference perspective finds that people in better health than their peers also report higher well-being than their peers. However, we know very little about whether changes in one domain are accompanied by changes in the other domain, particularly at the within-person level. In the present study, we introduce the construct of health sensitivity, that is, how susceptible an individuals’ well-being is to changes in physical health. In doing so, we used 9-wave longitudinal data covering 17 years from the Health and Retirement Study (N = 21,689; 50–109 year olds; 55% women) and applied multilevel modeling to examine the covariation of central indicators of well-being (depressive affect) and health (functional limitations) simultaneously at both the between-person and within-person level. At the within-person level, we found evidence of health sensitivity—on occasions when a typical person experienced more functional limitations than usual, he or she also reported more depressive affect—and that health sensitivity decreased with age. Survival analysis revealed that health sensitivity was related to mortality hazards, controlling for mean levels of health and well-being. We discuss the theoretical importance of examining within-person associations between health and well-being and consider practical implications.