The mental health of cancer survivors has not always been the primary emphasis of treatment protocols since physical health outcomes have taken precedence. Older cancer survivors experience a double jeopardy since they are at risk for memory impairments and mild cognitive impairment and because they are greater than 55 years of age. Of the 9.6 million cancer survivors in the US who have completed active treatment, many report cognitive difficulties, with labels such as "chemo brain," "not as sharp," "woolly-headedness," or the "mind does not work as quickly". To date, most of our knowledge of cognitive impairment in cancer survivors comes from female breast cancer survivors. Studies indicate that these survivors have diminished executive function, verbal memory, and motor function. Cancer survivors want to live independently in the community for as long as possible however, these cognitive deficits may prevent this desired lifestyle. To broaden our understanding this paper reviews the literature on the cognitive impairment and memory deficits experienced by three groups of cancer survivors breast, colorectal, and prostate cancer, that make up 60% of all survivors nationally. Even though mental health declined after a cancer diagnosis, the long-term outcomes of cancer survivors did not differ from persons without cancer in depression or cognitive function.