This event-related fMRI study examined the impact of processing load on the BOLD response to emotional expressions. Participants were presented with composite stimuli consisting of neutral and fearful faces upon which semi-transparent words were superimposed. This manipulation held stimulus-driven features constant across multiple levels of processing load. Participants made either (1) gender discriminations based on the face; (2) case judgments based on the words; or (3) syllable number judgments based on the words. A significant main effect for processing load was revealed in prefrontal cortex, parietal cortex, visual processing areas, and amygdala. Critically, enhanced activity in the amygdala and medial prefrontal cortex seen during gender discriminations was significantly reduced during the linguistic task conditions. A connectivity analysis conducted to investigate theories of cognitive modulation of emotion showed that activity in dorsolateral prefrontal cortex was inversely related to activity in the ventromedial prefrontal cortex. Together, the data suggest that the processing of task-irrelevant emotional information, like neutral information, is subject to the effects of processing load and is under top-down control.