Global rates of infertility are significant, especially in many developing countries. In developed countries, infertility affects up to 15% of the reproductive-age population, with male factors involved in 50% of cases. During medical evaluation, a review of current and past medications and social habits is essential, as they may adversely affect fertility. In addition, knowledge of exposure is important for counseling regarding future fertility, as many such toxic exposures have reversible effects. We reviewed the English literature over the past 35 years for all types of studies that have examined drug, medication, and other exposures that have been linked to altered semen quality and male fertility. Both human and animal studies were included in this analysis. The goal of this review is to delineate the medication and habitual risk factors that have been associated with male infertility.