This study explored the relationships among multimodal imaging, clinical features, and language impairment in patients with left temporal lobe epilepsy (LTLE). Fourteen patients with LTLE and 26 controls underwent structural MRI, functional MRI, diffusion tensor imaging, and neuropsychological language tasks. Laterality indices were calculated for each imaging modality and a principal component (PC) was derived from language measures. Correlations were performed among imaging measures, as well as to the language PC. In controls, better language performance was associated with stronger left-lateralized temporo-parietal and temporo-occipital activations. In LTLE, better language performance was associated with stronger right-lateralized inferior frontal, temporo-parietal, and temporo-occipital activations. These right-lateralized activations in LTLE were associated with right-lateralized arcuate fasciculus fractional anisotropy. These data suggest that interhemispheric language reorganization in LTLE is associated with alterations to perisylvian white matter. These concurrent structural and functional shifts from left to right may help to mitigate language impairment in LTLE.