Face adaptation paradigms have been used extensively to investigate the mechanisms underlying the processing of several different facial characteristics including face shape, identity, view and emotional expression. Judgements of facial trustworthiness can also be influenced by visual adaptation; to date these (un)trustworthy face aftereffects have only been shown following adaptation to emotional expression and facial masculinity/femininity. In this study we assessed how exposure to trustworthy and untrustworthy faces influenced the perception of the trustworthiness of subsequent test faces. In a mixed factorial design experiment, we tested the influence of adaptation to female and male faces on the perception of subsequent female and male faces in both female and male observers. In female observers, we found that following adaptation to trustworthy and untrustworthy faces subsequent test faces appeared less like the adapting stimuli. Sex of the adapting and test faces did not have significant influence on these (un)trustworthy face aftereffects. In male observers, however, we found no significant influence of the effect of adaptation on the subsequent perception of face trustworthiness. The clear difference in the visual aftereffects induced in female and male observers indicates the operation of different mechanisms underlying the perception of facial trustworthiness, and future studies should investigate these mechanisms separately in female and male observers.