The question how fast spatial attention moves between different visual objects remains debated. We used electrophysiological measures to determine the speed of voluntary and visually guided shifts of attention. Participants shifted attention from a known benchmark object (T1) to a benchmark-defined target object (T2) in tasks where these shifts had to be controlled endogenously and tasks where they could be guided by visible stimulus attributes (target features or arrow cues). To track the speed of these attention shifts, we measured event-related potential (ERP) markers of attentional object selection (N2pc components). The N2pc to T1 emerged earlier than the N2pc to T2, confirming the presence of serial attention shifts. N2pc onset differences between T1 and T2 revealed that shifts guided by target features were triggered within 50 ms, whereas voluntary movements of attention took substantially longer (150 ms). Attention shifts signaled by arrow cues were initiated within about 100 ms. Results show that genuinely voluntary shifts of attention are slower than shifts that are guided by cues or target features, but can still be initiated more rapidly than has previously been assumed. They also demonstrate that EEG markers can track different types of serial attentional selection processes with high temporal precision.