Visual processing of natural scenes is carried out in a hierarchical sequence of stages that involve the analysis of progressively more complex features of the visual input. Recent studies have suggested that the semantic content of natural stimuli (e.g., real world photos) can be categorized based on statistical regularities in their appearance, which can be detected early in the visual processing stream. Here we review the studies which have investigated the role of scene statistics in the perception of natural scenes, focusing on both basic visual processing and specific tasks (visual search, expert categorization, emotional picture viewing). Visual processing seems to be adapted to visual regularities in the visual input, such as the amplitude-frequency relationship. Moreover, scene statistics can aid performance in specific tasks such as distinguishing animals from artifactual scenes, possibly by modulating early visual processing stages.