When objects in a visual scene are positioned in close proximity, eye movements to these objects tend to land at an intermediate location between the objects (i.e. the global effect). This effect is most pronounced for short latency saccades and is therefore believed to be reflexive and dominantly controlled by bottom-up information. At longer latencies this effect can be modulated by top-down factors. The current study established the time course at which top-down information starts to have an influence on bottom-up averaging. In a standard global effect task two peripheral stimuli (a red and a green abrupt onset) were positioned within an angular distance of 20°. In the condition in which observers received no specific target instruction, the eyes landed in between the red and green element establishing the classic global effect. However, when observers were instructed to make a saccade to the red element during a whole block or when the target color varied from trial-to-trial (red or green), a clear effect of the target instruction on the accuracy of the landing position of the primary saccade was found. With increasing saccade latencies, the eyes landed closer to the instructed target. Crucially, however, this effect was even seen for the shortest saccade latencies (as early as 200 ms), suggesting that saccade averaging is affected early on by top-down processes.