Infertility is generally defined as a couple's inability to conceive after 1 year of unprotected intercourse. When infertile couples seek assistance, a male factor will be identified half of the time. Once the male has been evaluated, there are four main categories to describe his infertility: (1) idiopathic, (2) post-testicular/obstructive, (3) primary–where the Sertoli and/or Leydig cells of the testis fail, and (4) secondary–where there is a problem with the hypothalamus and/or pituitary. The last, hypogonadotropic hypogonadism (HH), accounts for up to 2% of infertile men. HH is either congenital or acquired and usually can be successfully treated by medical intervention. This review will focus on the hypothalamus-pituitary-gonadal axis, specific defects of this coordination center, and potential interventions for improving male-factor fertility.