The multiple-demand (MD) network is sensitive to many aspects of task difficulty, including such factors as rule complexity, memory load, attentional switching and inhibition. Many accounts link MD activity to top-down task control, raising the question of response when performance is limited by the quality of sensory input, and indeed, some prior results suggest little effect of sensory manipulations. Here we examined judgments of motion direction, manipulating difficulty by either motion coherence or salience of irrelevant dots. We manipulated each difficulty type across six levels, from very easy to very hard, and additionally manipulated whether difficulty level was blocked, and thus known in advance, or randomized. Despite the very large manipulations employed, difficulty had little effect on MD activity, especially for the coherence manipulation. Contrasting with these small or absent effects, we observed the usual increase of MD activity with increased rule complexity. We suggest that, for simple sensory discriminations, it may be impossible to compensate for reduced stimulus information by increased top-down control.