Mild cognitive impairment (MCI), which shows high risk for conversion to Alzheimer's disease (AD), is accompanied by progressive visual deteriorations that so far are poorly understood. Here, we compared dorsal and ventral visual stream functional magnetic resonance imaging (fMRI) activity among amnestic MCI, healthy elderly, and young participants during structure-from-motion (SFM) face categorization performance. Task performance varied with stimulus depth and duration levels and differences among groups were highly correlated with face-related fMRI activation patterns. Young participants showed larger activation to faces than scrambled faces (face sensitivity) in the right fusiform face area (FFA) and right occipital face area (OFA) whereas in elderly, this difference was reduced. Surprisingly, in MCI, scrambled faces elicited larger activation in right FFA/OFA than faces. The latter observation may be related to the additional finding of elevated depth sensitivity in left FFA/OFA of MCI, suggesting that an increased representation of low-level stimulus aspects may impair face perception in MCI. Discriminant function analysis using face and depth sensitivity indices in FFA/OFA classified MCI and healthy elderly with 88.2% accuracy, marking a fundamental distinction between groups. Potentially related findings include altered activation patterns in dorsal–ventral stream integration regions and attention-related networks of MCI patients. Our results highlight aberrant visual and additional potentially compensatory processes that identify dispositions of (preclinical) AD.