Women with breast cancer can experience persisting cognitive deficits post treatment. We conducted a multilevel meta-analysis of cognitive function in survivors treated with chemotherapy (Ch+) to estimate the magnitude of cognitive impairment relative to healthy (HC) and chemo-negative (Ch−) controls. Seventy-two studies published up to October 2016 involving 2939 Ch+ yielded 1594 effect sizes. Ch+ demonstrated overall cognitive impairment in comparison with HC but not with Ch−. Relative to HC, Ch+ showed impairment in attention/concentration, processing speed, language, immediate recall, delayed recall, and executive function. Deficits in memory recall and executive function remained significant after adjusting for prechemotherapy group differences. Ch+ performed worse than Ch− in attention/concentration and executive function, and the groups performed equivalently after accounting for prechemotherapy neurocognitive differences. These results demonstrate that cognitive deficits in Ch+ depend in large part on the comparison group, the cognitive domains examined, and whether prechemotherapy baseline neurocognition is measured. Cancer and/or other treatment-related factors contribute to subtle memory recall and executive function impairments in breast cancer survivors.