Humans often evaluate their abilities by comparing their personal performance with that of others. For this process, it is critical whether the comparison turns out in one’s favor or against it. Here, we investigate how social comparisons of performance are encoded and integrated on the neural level. We collected functional magnetic resonance images while subjects answered questions in a knowledge quiz that was related to their profession. After each question, subjects received a feedback about their personal performance, followed by a feedback about the performance of a reference group who had been quizzed beforehand. Based on the subjects’ personal performance, we divided trials in downward and upward comparisons. We found that upward comparisons correlated with activity in the dorsolateral prefrontal cortex and the anterior insula. Downward comparisons were associated with increased activation in the ventral striatum (VS), the medial orbitofrontal cortex and the ventral anterior cingulate cortex (ACC). The extent to which subjects outperformed the reference group modulated the activity in the VS and in the dorsal ACC. We suggest that the co-activation of the VS and the dorsal ACC contributes to the integration of downward comparisons into the evaluation of personal performance.