This study uses current epidemiological data to evaluate whether phytoestrogen intake is associated with a reduced risk of prostate cancer. We performed a random-effect meta-analysis of published data retrieved from PubMed, Web of Science, ProQuest, and CNKI, which was supplemented by a manual search of relevant references. Study quality was assessed using the Newcastle–Ottawa Scale (NOS). Subgroup analysis and meta-regression were performed to explore the source of heterogeneity. Sensitivity analysis was evaluated to assess the stability of the results. Egger's test and funnel plots were used to detect the existence of publication bias. We retrieved 507 papers, and 29 studies were ultimately confirmed as eligible. The meta-analysis showed that phytoestrogen intake was significantly associated with a reduced risk of prostate cancer, with an odds ratio (OR) of 0.77 (95% CI 0.66–0.88; I2 = 77.6%). The food/nutritional sources that were significantly associated with a reduced risk of prostate cancer included soy and soy products, tofu, legumes, daidzein, and genistein. Subgroup analysis indicated that the associations were significant among Asians and Caucasians, but not among Africans. Meta-regression revealed that the pooled OR increased with the number of cases in the studies. The results might be affected by publication bias based on the Eggers' test (p = 0.011) and the asymmetry of the funnel plot. Phytoestrogen intake may reduce the risk of prostate cancer in Asians and Caucasians. Regular intake of food that is rich in phytoestrogens, such as soy/soy products or legumes, should be recommended.