It has been long proposed that our extraordinary face recognition ability stems from holistic face processing. Two widely-used behavioral hallmarks of holistic face processing are the whole-part effect (WPE) and composite-face effect (CFE). However, it remains unknown whether these two effects reflect similar or different aspects of holistic face processing. Here we investigated this question by examining whether the WPE and CFE involved shared or distinct neural substrates in a large sample of participants (N=200). We found that the WPE and CFE showed hemispheric dissociation in the fusiform face area (FFA), that is, the WPE was correlated with face selectivity in the left FFA, while the CFE was correlated with face selectivity in the right FFA. Further, the correlation between the WPE and face selectivity was largely driven by the FFA response to faces, whereas the association between the CFE and face selectivity resulted from suppressed response to objects in the right FFA. Finally, we also observed dissociated correlation patterns of the WPE and CFE in other face-selective regions and across the whole brain. These results suggest that the WPE and CFE may reflect different aspects of holistic face processing, which shed new light on the behavioral dissociations of these two effects demonstrated in literature.