Memory impairment is common and a cause of unmet need after stroke. One third of patients recover spontaneously over the first year. Others develop later decline. The mechanism of recovery and the contribution of comorbid neurodegenerative pathology are not well understood. Data from older adults and Mild Cognitive Impairment suggest preserved memory depends on reorganisation of function between the fornix and other temporal lobe white matter tracts. The STRATEGIC study investigates memory over the first year post-stroke. The current analysis uses baseline data to investigate the contributions of initial infarct and white matter status to initial memory impairment. Patients (n = 21; 5 lacunar, 12 MCA, 4 PCA; 11 left hemisphere, 10 right) performed a cognitive battery and had diffusion-weighted MRI at 30-95 days after stroke onset (median = 64 days). Healthy controls (n = 33) provided the same measures. We reconstructed the fornix using HARDI-based deterministic tractography. Lesions were drawn manually on FLAIR images and direct injury of the fornix was excluded. We tested free recall using a delayed verbal memory test and recognition memory using a face recognition test with trialwise confidence ratings. Recall was markedly lower in patients than controls (50% vs 71%, p < 0.001) but did not correlate with age, lesion volume or fornix integrity in patients. Recognition memory was unimpaired in patients. For confidently remembered items, performance correlated with fornix mean diffusivity in both controls (r = -0.344, p < 0.05) and patients (r = -0.544, p < 0.05). Controlling for age eliminated the correlation in controls but in patients there was a correlation independent of age and lesion volume (r = -0.531, p < 0.05). These findings demonstrate that preserved recognition memory after stroke depends on the status of the fornix and is independent of infarct volume. Compromise of fornix structure from pre-existing neurodegeneration may be a predictor of poor cognitive outcome after stroke. Verbal recall in patients was independent of fornix integrity. Investigation of other temporal lobe pathways may reveal reorganisation underpinning partially preserved recall in some patients.