How limited representational capacity is divided when multiple items need to be processed simultaneously is a fundamental question in cognitive psychology. The double-target deficit is the finding that, when monitoring multiple locations or information streams for targets, identification of 2 simultaneous targets is substantially worse than is predicted from the cost of divided attention alone. This finding suggests that targets and nontargets are treated differently by the cognitive system. We investigated the double-target deficit in 4 different visual decision tasks using noisy, backwardly masked targets presented for a range of exposure durations to test the theory that the deficit reflects a capacity limitation of visual short-term memory (VSTM). We quantified the deficit using a sample-size model of VSTM and 2 different models of the decision process: a signal detection MAX model and an optimum likelihood ratio model. We found a double-target deficit in all 4 tasks which increased in magnitude for briefer displays, consistent with the capacity limits of VSTM. We explained the exposure dependency using a competitive interaction model in which nontargets compete for access to VSTM at a slower rate than targets. Our findings support 2-stage models of visual processing in which the most target-like stimuli gain priority access into VSTM before the decision process begins.