Visual field or retinotopic mapping is one of the most frequently used paradigms in fMRI. It uses activity evoked by position-varying high luminance contrast visual patterns presented throughout the visual field for determining the spatial organization of cortical visual areas. While the advantage of using high luminance contrast is that it tends to drive a wide range of neural populations – thus resulting in high signal-to-noise BOLD responses – this may also be a limitation, especially for approaches that attempt to squeeze more information out of the BOLD response, such as population receptive field (pRF) mapping. In that case, more selective stimulation of a subset of neurons – despite reduced signals – could result in better characterization of pRF properties. Here, we used a second-order stimulus based on local differences in orientation texture – to which we refer as orientation contrast – to perform retinotopic mapping. Participants in our experiment viewed arrays of Gabor patches composed of a foreground (a bar) and a background. These could only be distinguished on the basis of a difference in patch orientation. In our analyses, we compare the pRF properties obtained using this new orientation contrast-based retinotopy (OCR) to those obtained using classic luminance contrast-based retinotopy (LCR). Specifically, in higher order cortical visual areas such as LO, our novel approach resulted in non-trivial reductions in estimated population receptive field size of around 30%. A set of control experiments confirms that the most plausible cause for this reduction is that OCR mainly drives neurons sensitive to orientation contrast. We discuss how OCR – by limiting receptive field scatter and reducing BOLD displacement – may result in more accurate pRF localization as well. Estimation of neuronal properties is crucial for interpreting cortical function. Therefore, we conclude that using our approach, it is possible to selectively target particular neuronal populations, opening the way to use pRF modeling to dissect the response properties of more clearly-defined neuronal populations in different visual areas.