Objective: Previous studies have shown that the e4-allele of the APOE gene is associated with a higher risk of developing dementia. Our study investigated whether well-known associations between lifestyle factors and cognitive functioning may be stronger in individuals who carry the dementia risk variant of the APOE gene and whether this association is amplified with older age. Method: Data analysis comprised 7,526 participants (aged 40- to 79-years-old) from the population-based LIFE-Adult-study. The effect of the APOE e4-allele on the association between lifestyle factors (smoking, physical activity, being overweight, occupational attainment) and cognitive performance (trail making test [TMT] B, verbal fluency test [VFT]) was analyzed via multivariate generalized linear modeling adjusted for APOE e2-allele, age, gender, education, stroke, and heart attack. Results: Smoking, less physical activity, and lower occupational attainment was associated with a poorer performance in the TMT B and VFT. Neither the APOE e4-allele nor interactions with the APOE e4-allele were significantly associated with cognitive performance. The association between physical activity and occupational attainment on performance in the TMT B were stronger in older age, but the APOE gene did not modify those associations. Conclusions: Our findings suggest that the dementia risk variant of the APOE gene does not alter the association between lifestyle factors and cognitive performance in the general population aged 40- to 79-years-old. However, as lifestyle factors impact cognitive aging, research efforts should focus on establishing effective interventions promoting healthy lifestyle behaviors to counteract adverse cognitive aging processes.