Genetics, early experience, and culture shape caregiving, but it is still not clear how genetics, early experiences, and cultural factors might interact to influence specific caregiving propensities, such as adult responsiveness to infant cues. To address this gap, 80 Italian adults (50% M; 18–25 years) were (1) genotyped for two oxytocin receptor gene polymorphisms (rs53576 and rs2254298) and the serotonin transporter gene polymorphism (5-HTTLPR), which are implicated in parenting behaviour, (2) completed the Adult Parental Acceptance/Rejection Questionnaire to evaluate their recollections of parental behaviours toward them in childhood, and (3) were administered a Single Category Implicit Association Test to evaluate their implicit responses to faces of Italian infants, Japanese infants, and Italian adults. Analysis of implicit associations revealed that Italian infant faces were evaluated as most positive; participants in the rs53576 GG group had the most positive implicit associations to Italian infant faces; the serotonin polymorphism moderated the effect of early care experiences on adults’ implicit association to both Italian infant and adult female faces. Finally, 5-HTTLPR S carriers showed less positive implicit responses to Japanese infant faces. We conclude that adult in-group preference extends to in-group infant faces and that implicit responses to social cues are influenced by interactions of genetics, early care experiences, and cultural factors. These findings have implications for understanding processes that regulate adult caregiving.