Several recent studies have suggested that sperm concentrations and semen quality have been decreasing over the past several decades in many areas of the world. The etiology of these decreases is currently unknown. Acute events can have significant impacts on spermatogenesis and are often readily identified during the male fertility evaluation. The majority of male factor infertility, however, is idiopathic. Chronic, low-dose exposures to chemicals and nutrients are more difficult to identify, but are extremely prevalent. These exposures have been shown to have dramatic effects on both individual and community health and interest in the cumulative and synergistic impacts of such agents on spermatogenesis has been increasing. While our understanding of these potential hazards is evolving, it is clear that they may significantly influence male reproductive potential. This review explores the literature related to effects of chronic exposures from drug use, dietary intake, and the environment on spermatogenesis in humans and animals.