Complementary neurophysiological recordings in rhesus macaques (Macaca mulatta) and functional neuroimaging in human subjects show that the primary taste cortex in the rostral insula and adjoining frontal operculum provides separate and combined representations of the taste, temperature and texture (including viscosity and fat texture) of food in the mouth independently of hunger and thus of reward value and pleasantness. One synapse on, in the orbitofrontal cortex, these sensory inputs are for some neurons combined by learning with olfactory and visual inputs. Different neurons respond to different combinations, providing a rich representation of the sensory properties of food. In the orbitofrontal cortex feeding to satiety with one food decreases the responses of these neurons to that food, but not to other foods, showing that sensory-specific satiety is computed in the primate (including the human) orbitofrontal cortex. Consistently, activation of parts of the human orbitofrontal cortex correlates with subjective ratings of the pleasantness of the taste and smell of food. Cognitive factors, such as a word label presented with an odour, influence the pleasantness of the odour, and the activation produced by the odour in the orbitofrontal cortex. Food intake is thus controlled by building a multimodal representation of the sensory properties of food in the orbitofrontal cortex and gating this representation by satiety signals to produce a representation of the pleasantness or reward value of food that drives food intake. Factors that lead this system to become unbalanced and contribute to overeating and obesity are described.