In visual search, relatively infrequent targets are more likely to be “missed,” a phenomenon known as the low-prevalence effect (LPE). Across five experiments, we examined the LPE in unfamiliar face matching, focusing on the roles of feedback and criterion shifting. Across experiments, observers made identity match/mismatch decisions to photograph pairs, and we manipulated target (i.e., identity mismatch) prevalence. Experiment 1 established the necessity of feedback for the LPE; observers were only sensitive to prevalence disparities when provided trial-level feedback. In Experiment 2, target prevalence affected decision criteria, without concomitant effects on perceptual sensitivity. In Experiments 3 through 5, we adopted a “retraining” paradigm, in which observers encountered blocks of high-prevalence targets midway through four 50-trial face-matching quartiles. High-prevalence targets were visually obvious (Experiment 3) or less obvious (Experiments 4 and 5). Whereas observers in equal-prevalence conditions remained unbiased throughout the experiments, those in low-prevalence conditions adopted conservative criteria by the second quartile. This criterion shift was largely resistant to “unbiasing” efforts. Only Experiment 5, which used an 18-item retraining block, revealed a successful (albeit slight) third-quartile liberal criterion shift, but observers were strongly conservative again by the fourth quartile. We discuss the applied and theoretical consequences of these results.