In the past 2 decades, it has become evident that individuals born with congenital heart disease (CHD) are at risk of developing life-long neurological deficits. Multifactorial risk factors contributing to neurodevelopmental abnormalities associated with CHD have been identified; however, the underlying causes remain largely unknown, and efforts to address this issue have only recently begun. There has been a dramatic shift in focus from newly acquired brain injuries associated with corrective and palliative heart surgery to antenatal and preoperative factors governing altered brain maturation in CHD. In this review, we describe key time windows of development during which the immature brain is vulnerable to injury. Special emphasis is placed on the dynamic nature of cellular events and how CHD may adversely impact the cellular units and networks necessary for proper cognitive and motor function. In addition, we describe current gaps in knowledge and offer perspectives about what can be done to improve our understanding of neurological deficits in CHD. Ultimately, a multidisciplinary approach will be essential to prevent or improve adverse neurodevelopmental outcomes in individuals surviving CHD.