We investigated the brain network involved in speech sensorimotor processing by studying patients with post-stroke aphasia using an altered auditory feedback (AAF) paradigm. We combined lesion-symptom-mapping analysis and behavioral testing to examine the pervasiveness of speech sensorimotor deficits and their relationship with cortical damage. Sixteen participants with aphasia and sixteen neurologically intact individuals completed a speech task under AAF. The task involved producing speech vowel sounds under the real-time pitch-shifted auditory feedback alteration. This task provided an objective measure for each individual's ability to compensate for mismatch (error) in speech auditory feedback. Results indicated that compensatory speech responses to AAF were significantly diminished in participants with aphasia compared with control. We observed that within the aphasic group, subjects with lower scores on the speech repetition task exhibited greater degree of diminished responses. Lesion-symptom-mapping analysis revealed that the onset phase (50–150 ms) of diminished AAF responses were predicted by damage to auditory cortical regions within the superior and middle temporal gyrus, whereas the rising phase (150–250 ms) and the peak (250–350 ms) of diminished AAF responses were predicted with damage to the inferior frontal gyrus and supramarginal gyrus areas, respectively. These findings suggest that damage to the auditory, motor, and auditory-motor integration networks are associated with impaired sensorimotor function for speech error processing. We suggest that a sensorimotor integration network, as revealed by brain regions related to temporal specific components of AAF responses, is related to speech processing and specific aspects of speech impairment, notably repetition deficits, in individuals with aphasia.