Using a case-control design, we evaluated differences in risk factors for colorectal polyps according to histological type, anatomical site, and severity. Participants were enrollees in the Group Health Cooperative aged 20–79 years who underwent colonoscopy in Seattle, Washington, between 1998 and 2007 and comprised 628 adenoma cases, 594 serrated polyp cases, 247 cases with both types of polyps, and 1,037 polyp-free controls. Participants completed a structured interview, and polyps were evaluated via standardized pathology review. We used multivariable polytomous logistic regression to compare case groups with controls and with the other case groups. Factors for which the strength of the association varied significantly between adenomas and serrated polyps were sex (P < 0.001), use of estrogen-only postmenopausal hormone therapy (P = 0.01), and smoking status (P < 0.001). For lesion severity, prior endoscopy (P < 0.001) and age (P = 0.05) had significantly stronger associations with advanced adenomas than with nonadvanced adenomas; and higher education was positively correlated with sessile serrated polyps but not with other serrated polyps (P = 0.02). Statistically significant, site-specific associations were observed for current cigarette smoking (P = 0.05 among adenomas and P < 0.001 among serrated polyps), postmenopausal estrogen-only therapy (P = 0.01 among adenomas), and obesity (P = 0.01 among serrated polyps). These findings further illustrate the epidemiologic heterogeneity of colorectal neoplasia and may help elucidate carcinogenic mechanisms for distinct pathways.