A central question in prescribing opioids for chronic non-cancer pain (CNCP) is how to best balance the risk of opioid abuse and dependence with the benefits of pain relief. To achieve this balance, clinicians need an understanding of the risk factors for opioid abuse, an issue that is only partially understood. We conducted a secondary data analysis of regional VA longitudinal administrative data (years 2000–2005) for chronic users of opioids for CNCP (n = 15,160) to investigate risk factors for the development of clinically recognized (i.e., diagnosed) opioid abuse or dependence among these individuals. We analyzed four broad groups of possible risk factors: (i) non-opioid substance abuse disorders, (ii) painful physical health disorders, (iii) mental health disorders, and (iv) socio-demographic factors. In adjusted models, a diagnosis of non-opioid substance abuse was the strongest predictor of opioid abuse/dependence (OR = 2.34, p < 0.001). Mental health disorders were moderately strong predictors (OR = 1.46, p = 0.005) of opioid abuse/dependence. However, the prevalence of mental health disorders was much higher than the prevalence of non-opioid substance abuse disorders (45.3% vs. 7.6%) among users of opioids for CNCP, suggesting that mental health disorders account for more of the population attributable risk for opioid abuse than does non-opioid substance abuse. Males, younger adults, and individuals with greater days supply of prescription opioids dispensed in 2002 were more likely to develop opioid abuse/dependence. Clinicians need to carefully screen for substance abuse and mental health disorders in candidates for opioid therapy and facilitate appropriate treatment of these disorders.