Visual neglect is a disabling consequence of right hemisphere damage, whereby patients fail to detect left-sided objects. Its precise mechanisms are debated, but there is some consensus that distinct component deficits may variously associate and interact in different patients. Here we used a touch-screen based procedure to study two putative component deficits of neglect, rightward “magnetic” attraction of attention and impaired spatial working memory, in a group of 47 right brain-damaged patients, of whom 33 had signs of left neglect. Patients performed a visual search task on three distinct conditions, whereby touched targets could (1) be tagged, (2) disappear or (3) show no change. Magnetic attraction of attention was defined as more left neglect on the tag condition than on the disappear condition, where right-sided disappeared targets could not capture patients' attention. Impaired spatial working memory should instead produce more neglect on the no change condition, where no external cue indicated that a target had already been explored, than on the tag condition. Using a specifically developed analysis algorithm, we identified significant differences of performance between the critical conditions. Neglect patients as a group performed better on the disappear condition than on the no change condition and also better in the tag condition comparing with the no change condition. No difference was found between the tag condition and the disappear condition. Some of our neglect patients had dissociated patterns of performance, with predominant magnetic attraction or impaired spatial working memory. Anatomical results issued from both grey matter analysis and fiber tracking were consistent with the typical patterns of fronto-parietal and occipito-frontal disconnection in neglect, but did not identify lesional patterns specifically associated with one or another deficit, thus suggesting the possible co-localization of attentional and working memory processes in fronto-parietal networks. These findings give support to the hypothesis of the co-occurrence of distinct cognitive deficits in visual neglect and stress the necessity of multi-component models of visuospatial disorders.