This study aims to investigate the diffusion metrics of left versus right temporal lobe epilepsy in a well-defined subgroup of patients with mesial temporal lobe epilepsy (mTLE) because of unilateral hippocampal sclerosis while taking into account interhemispheric differences. Eighteen patients with TLE [nine left temporal lobe epilepsy (LTLE) and nine right temporal lobe epilepsy (RTLE)] and a norm group of 36 nonepileptic individuals were scanned with a multiband accelerated diffusion tensor imaging protocol at 3T. The scalar diffusion tensor parameters fractional anisotropy (FA), mean diffusivity (MD), and radial diffusivity (RD) and, after projection on a symmetric skeleton, their hemispheric difference (dFA, dMD, and dRD) were analyzed using tract-based spatial statistics. In the cluster with significantly (P<0.008) different dFA, dMD, and dRD between right TLE and left TLE, the hemispheric difference in the mean scalar indices (dmFA, dmMD, and dmRD) was assessed and tested for differences using a one-way analysis of variance and for correlation with patient age, seizure onset, or duration of epilepsy using Pearson’s correlation. Patients with LTLE showed lower dFA, higher dMD, and higher dRD (P<0.008) compared with patients with RTLE in a cluster including parts of the uncinated and inferior longitudinal fasciculus and the inferior fronto-occipital fasciculus. dmFA, dmMD, and dmRD differed significantly between groups (P<10−3, corrected) and showed no correlation with patient age, seizure onset, or duration of epilepsy. The exclusion of bilateral interindividual variance through the calculation of the hemispheric difference of the diffusion metrics by the symmetric variant of tract-based spatial statistics allows for a sensitive differentiation of LTLE and RTLE with unilateral hippocampal sclerosis.