The need to promote fertiliser use by African smallholder farmers to counteract the current decline in per capita food production is widely recognised. But soil heterogeneity results in variable responses of crops to fertilisers within single farms. We used existing databases on maize production under farmer (F-M) and researcher management (R-M) to analyse the effect of soil heterogeneity on the different components of nutrient use efficiency by maize growing on smallholder farms in western Kenya: nutrient availability, capture and conversion efficiencies and crop biomass partitioning. Subsequently, we used the simple model QUEFTS to calculate nutrient recovery efficiencies from the R-M plots and to calculate attainable yields with and without fertilisers based on measured soil properties across heterogeneous farms. The yield gap of maize between F-M and R-M varied from 0.5 to 3 t grain ha−1 season−1 across field types and localities. Poor fields under R-M yielded better than F-M, even without fertilisers. Such differences, of up to 1.1 t ha−1 greater yields under R-M conditions are attributable to improved agronomic management and germplasm. The relative response of maize to N–P–K fertilisers tended to decrease with increasing soil quality (soil C and extractable P), from a maximum of 4.4-fold to −0.5-fold relative to the control. Soil heterogeneity affected resource use efficiencies mainly through effects on the efficiency of resource capture. Apparent recovery efficiencies varied between 0 and 70% for N, 0 and 15% for P, and 0 to 52% for K. Resource conversion efficiencies were less variable across fields and localities, with average values of 97 kg DM kg−1 N, 558 kg DM kg−1 P and 111 kg DM kg−1 K taken up. Using measured soil chemical properties QUEFTS over-estimated observed yields under F-M, indicating that variable crop performance within and across farms cannot be ascribed solely to soil nutrient availability. For the R-M plots QUEFTS predicted positive crop responses to application of 30 kg P ha−1 and 30 kg P ha−1 + 90 kg N ha−1 for a wide range of soil qualities, indicating that there is room to improve current crop productivity through fertiliser use. To ensure their efficient use in sub-Saharan Africa mineral fertilisers should be: (1) targeted to specific niches of soil fertility within heterogeneous farms; and (2) go hand-in-hand with the implementation of agronomic measures to improve their capture and utilisation.