Many morphological and dynamic properties of the common carotid artery (CCA), e.g. lumen diameter, distension and wall thickness, can be measured non-invasively with ultrasound (US) techniques. As common to other medical image segmentation processes, this requires as a preliminary step the manual recognition of the artery of interest within the ultrasound image. In real-time US imaging, such manual initialization procedure interferes with the difficult task of the sonographer to select and maintain a proper image scan plane. Even for off-line US segmentation the requirement for human supervision and interaction precludes full automation. To eliminate user interference and to speed up processing for both real-time and off-line applications, we developed an algorithm for the automatic artery recognition in longitudinal US scans of the CCA. It acts directly on the envelopes of received radio frequency echo signals, eventually composing the ultrasound image. In order to properly exploit the information content of the arterial structure the envelopes are decimated, according to the two-dimensional resolution characteristics of the echo system, thereby substantially decreasing computational load. Subsequently, based upon the expected diameter range and a priori knowledge of the typical pattern in the echo envelope of the arterial wall–lumen complex, parametrical template matching is performed, resulting in the location of the lumen position along each echo line considered. Finally, in order to reject incorrect estimates, a spatial and temporal clustering method is applied. Adequate values for the parameters involved in the processing are obtained via off-line testing of the proposed algorithm on 128 echo data recordings from 45 subjects. Using those robust parameter values, correct and fast recognition of the artery is achieved in more than 98% of the 6185 processed frames. Since these results are obtained via rigorous data decimation and using a cascade of rather simple steps, the proposed automatic algorithm is suitable for real-time recognition of the CCA.