Open educational resources (OERs) enable anyone, anywhere, to access psychological science for free. Are OERs as effective learning tools as standard textbooks (STBs)? I compared students using an OER with students using STBs in 2 large, multisite studies (N = 1,099 and N = 2,229, respectively). Studies measure key student variables and possible confounds such as metacognition, study techniques, time spent studying, perceptions of the instructor, and rating of the quality and helpfulness of the textbook. Students completed a standardized test using a subset of items from a released Advanced Placement exam. In both studies, OER users reported lower ACT scores. In Study 1, students using an OER scored lower on the test after controlling for ACT scores, F(1, 777) = 48.09, p < .001, ηp2 = .06, and rated the book as lower in quality. Scores on book helpfulness did not vary between groups. Study 2 also compared book format (hard copy or electronic) and showed a main effect of book used, no main effect of format, and a significant book used by format interaction effect, F(2, 1406) = 5.29, p = .005, ηp2 = .01: OER hard copy users scored lowest. Book used predicted significant variance in learning over and above ACT scores and student variables (STB users performed better). The results highlight limitations of current attempts to assess learning in psychology and underline the need for robust comparisons of a wider variety of OERs, with a focus on lower ability students, and e-book usage patterns.