Multiple lines of research indicate that fathers often treat boys and girls differently in ways that impact child outcomes. The complex picture that has emerged, however, is obscured by methodological challenges inherent to the study of parental caregiving, and no studies to date have examined the possibility that gender differences in observed real-world paternal behavior are related to differential paternal brain responses to male and female children. Here we compare fathers of daughters and fathers of sons in terms of naturalistically observed everyday caregiving behavior and neural responses to child picture stimuli. Compared with fathers of sons, fathers of daughters were more attentively engaged with their daughters, sang more to their daughters, used more analytical language and language related to sadness and the body with their daughters, and had a stronger neural response to their daughter’s happy facial expressions in areas of the brain important for reward and emotion regulation (medial and lateral orbitofrontal cortex [OFC]). In contrast, fathers of sons engaged in more rough and tumble play (RTP), used more achievement language with their sons, and had a stronger neural response to their son’s neutral facial expressions in the medial OFC (mOFC). Whereas the mOFC response to happy faces was negatively related to RTP, the mOFC response to neutral faces was positively related to RTP, specifically for fathers of boys. These results indicate that real-world paternal behavior and brain function differ as a function of child gender.