Accurately determining the familiarity of another and correctly establishing their identity are vital social skills. A considerable body of work has explored their perceptual and neural underpinnings and debate remains regarding whether they are dissociable, that is, separable parts of a dual process, or different aspects of a common retrieval process. Less is known about the specific visual information that guides familiarity judgments and how this compares to the information used to identify a face by name. Here we sought to establish the critical information underlying participants’ judgments of facial familiarity and identification. We created a new standardized stimulus set comprising 6 personally familiar and 12 unfamiliar faces and applied the Bubbles reverse-correlation methodology to establish the information driving correct performance in each task. Results revealed that markedly different information underlies familiarity and identity judgments. When categorizing familiarity, participants relied more upon lower spatial-frequency, broad facial cues (eye and face shape) than when categorizing identity, which relied on fine details in the internal features (eyes and mouth). These results provide novel evidence of qualitatively distinct information use in familiarity and identification judgments and emphasize the importance of considering the task set for participants and their processing strategy when investigating face recognition.