Among patients with epilepsy, atypical (rightward) language lateralization has been associated with left-handedness, a left seizure focus, an early age at seizure onset, and familial sinistrality, although these associations are not consistently observed. No study has examined all of these factors in relation to language lateralization in the same epilepsy sample, let alone in a sample comprised only of patients with temporal lobe epilepsy. Moreover, little consideration has been given in previous studies to how language lateralization might be influenced by the interplay between different factors, or how much unique variance in language lateralization is explained by each factor. The primary aim of this study was to examine the combined influences of handedness, side of seizure focus, age at seizure onset, and familial sinistrality on language lateralization in temporal lobe epilepsy patients. A secondary aim was to determine which factors uniquely contribute to the prediction of language lateralization. 162 patients with intractable temporal lobe epilepsy underwent functional MRI language mapping, from which language lateralization indexes were derived. Degree of handedness was measured via the Edinburgh Handedness Inventory. Main and 2-way interaction effects on language lateralization indexes were examined via linear regressions and Fisher exact tests. Significant effects were next examined in multiple regressions to identify unique predictors of language lateralization indexes. When examined in isolation in regressions, only left-handedness and a left seizure focus predicted atypical (rightward) language lateralization. These results, however, were qualified by interaction effects demonstrating that stronger left hand preference was associated with greater atypical language lateralization only among patients with a left seizure focus, an early or intermediate age at seizure onset, or no familial sinistrality. In follow-up multiple regressions, the interaction terms accounted for a significant amount of variance in language lateralization indexes above and beyond main effects. Additionally, side of seizure focus and its interaction with handedness uniquely predicted language lateralization indexes. Results indicate that degree of left-handedness is a marker of greater atypical (rightward) language lateralization in temporal lobe epilepsy but only in the context of seizure characteristics that have the potential to drive joint reorganization of language and hand preference (i.e., left seizure focus, or early or intermediate age at seizure onset) or in the absence of a genetic predisposition for left-handedness (i.e., no familial sinistrality). This study advances existing knowledge by illustrating how different factors combine to jointly affect language lateralization, and by identifying side of seizure focus and its interaction with handedness as unique predictors of language lateralization in temporal lobe epilepsy.