The association between the Y chromosome haplogroup D2 and risk of azoospermia and low sperm motility has been previously studied, and it was indicated that haplogroups DE (YAP lineage) are associated with prostate cancer risk in Japanese males. Our assumption had been that Y chromosome haplogroups may be associated with sex hormone levels, because sex hormones have been deemed responsible for spermatogenesis and carcinogenesis. In this study, we assessed the association between Y chromosome haplogroups and sex hormone levels, including those of testosterone, sex hormone-binding globulin (SHBG), follicle-stimulating hormone (FSH), luteinizing hormone (LH), inhibin-B, and calculated free testosterone (cFT), in 901 young men from the general Japanese population (cohort 1) and 786 Japanese men of proven fertility (cohort 2). We found that the haplogroup D2a1 was significantly associated with high LH levels in a combined analysis involving two cohorts (β = 0.068, SE = 0.025, p = 0.0075), following correction for multiple testing. To date, this result is the first evidence that implicates Y chromosome haplogroups in an association with sex hormone levels.