Children begin to establish lexical–semantic representations during their first year of life, resulting in a rapid growth of vocabulary around 18–24 months of age. The neural mechanisms underlying this initial ability to map words onto conceptual representations remain relatively unknown. In the present study, the electrophysiological underpinnings of these mechanisms are explored during the critical phase of lexical acquisition using a picture-word matching paradigm. Event-Related Potentials (ERPs) elicited by words (either congruous or incongruous with the previous picture context) and pseudo-words are investigated in 20-month-old toddlers (N = 20) and compared to those elicited in a sample of adults (N = 20), reflecting the final and efficient system, and a sample of toddlers at familial risk for language and learning impairment (LLI, N = 15). The results suggest that the architecture underlying spoken word representation and processing is constant throughout development, even if some differences between children and adults emerged. Interestingly, children seem to be faster than adults in processing incongruent words, probably because relying on a different and more superficial strategy. This early strategy does not seem to be present in children at risk for LLI. In addition, both groups of children do not show different and specific electrophysiological underpinnings in response to real but incongruent words and unknown words, suggesting that during the critical phase of lexical acquisition any potential word is processed in a similar way. Overall, children at risk for LLI turned out to be sensitive to verbal incongruity of the lexical-semantic context, although some differences from typically developing children emerged, reflecting slower processing and less automatic responses. Taken together, the findings of this study pave the way to further research to investigate these effects in clinical and at-risk populations with the general purpose of disentangling the underlying mechanisms of lexical acquisition, and potentially predicting later language (dis)abilities.