Emotionally charged items are often remembered better, whereas a paradoxical loss of specificity is found for associative emotional information (specific memory). The balance between specific and generalized emotional memories appears to show large individual differences, potentially related to differences in (the risk for) affective disorders that are characterized by ‘overgeneralized’ emotional memories. Here, we investigate the neural underpinnings of individual differences in emotional associative memory. A large group of healthy male participants were scanned while encoding associations of face-photographs and written occupational identities that were of either neutral (‘driver’) or negative (‘murderer’) valence. Subsequently, memory was tested by prompting participants to retrieve the occupational identities corresponding to each face. Whereas in both valence categories a similar amount of faces was labeled correctly with ‘neutral’ and ‘negative’ identities, (gist memory), specific associations were found to be less accurately remembered when the occupational identity was negative compared to neutral (specific memory). This pattern of results suggests reduced memory specificity for associations containing a negatively valenced component. The encoding of these negative associations was paired with a selective increase in medial prefrontal cortex activity and medial prefrontal–hippocampal connectivity. Individual differences in valence-specific neural connectivity were predictive of valence-specific reduction of memory specificity. The relationship between loss of emotional memory specificity and medial prefrontal–hippocampal connectivity is in line with the hypothesized role of a medial prefrontal–hippocampal circuit in regulating memory specificity, and warrants further investigations in individuals displaying ‘overgeneralized’ emotional memories.