People form impressions of others from multiple sources of information. Facial appearance is one such source and judgments based on facial appearance are made after minimal exposure to faces. A more reliable source of information is affective person learning based on others’ past actions. Here we investigated whether the effects of such appearance-independent learning on face evaluation emerge after rapid face exposure, a response deadline procedure, and a lack of explicit recognition of the faces. In three experiments, participants learned to associate novel faces with negative and positive behaviors, and then evaluated the faces presented on their own, without the behaviors. Even after extremely brief exposures (e.g., 35 ms), participants evaluated faces previously associated with negative behaviors more negatively than those associated with positive behaviors (Experiment 1). The learning effect persisted when participants were asked to evaluate briefly presented faces before a response deadline (Experiment 2), although the effect was diminished. Finally, although this learning effect increased as a function of face recognition (Experiment 3), it was present with only minimal recognition, suggesting that participants do not need to deliberately retrieve behavioral information for it to influence face evaluation. Together, the findings suggest that person learning unrelated to facial appearance is a powerful determinant of face evaluation.