Neurogenetics research has begun to advance our understanding of how genetic variation gives rise to individual differences in brain function, which, in turn, shapes behavior and risk for psychopathology. Despite these advancements, neurogenetics research is currently confronted by three major challenges: (1) conducting research on individual variables with small effects, (2) absence of detailed mechanisms, and (3) a need to translate findings toward greater clinical relevance. In this review, we showcase techniques and developments that address these challenges and highlight the benefits of a neurogenetics approach to understanding brain, behavior and psychopathology. To address the challenge of small effects, we explore approaches including incorporating the environment, modeling epistatic relationships and using multilocus profiles. To address the challenge of mechanism, we explore how non-human animal research, epigenetics research and genome-wide association studies can inform our mechanistic understanding of behaviorally relevant brain function. Finally, to address the challenge of clinical relevance, we examine how neurogenetics research can identify novel therapeutic targets and for whom treatments work best. By addressing these challenges, neurogenetics research is poised to exponentially increase our understanding of how genetic variation interacts with the environment to shape the brain, behavior and risk for psychopathology.