Research has demonstrated the clinical effectiveness of pre-exposure prophylaxis (PrEP) for HIV prevention, but little is known about how factors at the individual-, interpersonal-, community-, and structural levels impact PrEP use for black men who have sex with men (BMSM). We advance existing work by examining how all levels of the ecological framework must be addressed for PrEP to be successfully implemented as an effective HIV prevention approach. We interviewed 31 BMSM three times each and 17 community stakeholders once each; interviews were taped, transcribed, and analyzed using the constant comparative method. Factors that influence how BMSM experienced PrEP emerged across all levels of the ecological framework: At the individual level, respondents were wary of giving medication to healthy people and of the potential side-effects. At the interpersonal level, BMSM believed that PrEP use would discourage condom use and that PrEP should only be one option for HIV prevention, not the main option. At the community level, men described not trusting the pharmaceutical industry and described PrEP as an option for others, not for themselves. At the structural level, BMSM talked about HIV and sexuality-related stigmas and how they must overcome those before PrEP engagement. BMSM are a key population in the US National HIV/AIDS Strategy, yet few individuals believe that PrEP would be personally helpful. Our research indicates the urgent need to raise awareness and address structural stigma and policies that could be substantial barriers to the scale-up and implementation of PrEP-related services.