In two experiments, we showed that irrelevant numerical information influenced the speed of sentence-picture verification. Participants were asked to verify whether the concept mentioned in a sentence matched the object presented in a subsequent picture. Concurrently, the number word attached to the concept in the sentence and the quantity of objects presented in the picture were manipulated (numerical congruency). The number of objects varied from one to four. In Experiment 1, participants read statements such as three dogs. In Experiment 2, they read sentences such as three dogs were wandering in the street. In both experiments, the verification speed revealed the interaction between response and numerical congruency. The verification times for concept-object match were faster when there was also numerical congruence (compared with incongruence) between the number word and quantity. On the other hand, there was no difference between numerical congruence and incongruence when the concept and object mismatched. The results are interpreted as evidence for the symbol grounding of number words in perceptual representation of small quantities, that is, quantities falling in the subitization range.