Accumulating evidence suggests that humans spontaneously adopt each other’s visuospatial perspective (VSP), but many aspects about the underlying mechanisms remain unknown. The aim of this study was to investigate whether knowledge about another’s visual access systematically modulates spontaneous VSP-taking. In a spatial compatibility task, a participant and a confederate sat at a 90°angle to each other, with visual stimuli being aligned vertically for the participants and horizontally for the confederate. In this task, VSP-taking is reflected in a spatial compatibility effect in the participant, because stimulus–response compatibility occurs only if the participant takes the confederate’s perspective. We manipulated the visual access of the confederate during the task by means of glasses with adjustable shutters that allowed or prevented the confederate from seeing the visual stimuli. The results of 2 experiments showed that people only adopted their task partner’s VSP if that person had unhindered visual access to the stimuli. Provided that the confederate had visual access to the participant’s stimuli, VSP-taking occurred regardless of whether the confederate performed the same visual task as the participant (Experiment 1) or a different, auditory task (Experiment 2). The results suggest that knowledge about another’s visual access is pivotal for triggering spontaneous VSP-taking, whereas having the same task is not. We discuss the possibility that spontaneous VSP-taking can effectively facilitate spatial alignment processes in social interaction.