Several studies analyzing the ontogenetic origin of cerebral lateralization provide evidences for a genetic foundation of handedness in humans that is modulated by environmental influences. Since other forms of behavioral lateralization are less investigated, it is unclear as to how far different functions display similar heritability. But deeper knowledge is necessary to understand if and how developmental coupling of different functions is based on a shared genetic background or on the impact of environmental influences. Here, we investigated the heritability of language lateralization assessed with the dichotic listening task, as well as the heritability of cognitive control processes modulating performance in this task. Overall, 103 families consisting of both parents and offspring were tested with the non-forced, as well as the forced-right and forced-left condition of the forced attention dichotic listening task, implemented in the iDichotic smartphone app, developed at the University of Bergen, Norway. The results indicate that the typical right ear advantage in the dichotic listening task shows weak and non-significant heritability (h2 = 0.003; p = 0.98). In contrast, cognitive factors, like attention focus (forced right condition: h2 = 0.36; p < 0.01; forced left condition: h2 = 0.28; p < 0.05) and cognitive control (Gain forced right: h2 = 0.39; p < 0.01; Gain forced left: h2 = 0.49; p < 0.01) showed stronger and significant heritability. These findings indicate a variable dependence of different aspects of a cognitive function on heritability and implicate a major contribution of non-genetic influences to individual language lateralization.