For infants it is crucial to differentiate conspecifics from other animates in order to profoundly learn about the world, the self, and other people. The current study investigates brain correlates of a categorical human–animal distinction in infants of different age groups as well as in adults. Using the categorical oddball task (Pauen, Wahl, & Hoehl, 2011), we compared event-related potentials of 4- and 7-month-old infants with those of adults. A total of 100 different-looking pictures of animals and humans were presented in random sequence. In 80% of the trials exemplars of one category were shown (standards), and in 20% of the trials exemplars of the contrasting category were shown (oddballs). In 7-month-olds, an increased Nc response was found for oddballs as compared to standards, independent of the oddball category. Furthermore, amplitude of the P400 was increased in reaction to standards as compared to oddballs, when humans served as standards (i.e., in the human-standard condition). No corresponding ERP-effects were observed in 4-month-olds. Thus, while 7-month-olds showed signs of categorical differentiation on the neural level, 4-month-olds' ERPs suggest less stable category representations within the categorical oddball paradigm. In adults, we found an increased N1 amplitude for oddballs as compared to standards. Thus, adults’ sensitivity to the relative frequencies of the contrasted categories at the level of the N1 was comparable to infants’ Nc response at 7 months. Furthermore, in adults we found the N2 amplitude to reflect category-specific processing, with a consistently increased amplitude in reaction to animals.