Knowledge of the genetic basis of sexual ornaments is essential to understand their evolution through sexual selection. Although carotenoid-based ornaments have been instrumental in the study of sexual selection, given the inability of animals to synthesize carotenoids de novo, they are generally assumed to be influenced solely by environmental variation. However, very few studies have directly estimated the role of genes and the environment in shaping variation in carotenoid-based traits. Using long-term individual-based data, we here explore the evolutionary potential of a dynamic, carotenoid-based ornament (namely skin coloration), in male and female common kestrels. We first estimate the amount of genetic variation underlying variation in hue, chroma and brightness. After correcting for sex differences, the chroma of the orange-yellow eye ring coloration was significantly heritable (h2 ± SE = 0.40 ± 0.17), whereas neither hue (h2 = 0) nor brightness (h2 = 0.02) was heritable. Second, we estimate the strength and shape of selection acting upon chromatic (hue and chroma) and achromatic (brightness) variation and show positive and negative directional selection on female but not male chroma and hue, respectively, whereas brightness was unrelated to fitness in both sexes. This suggests that different components of carotenoid-based signals traits may show different evolutionary dynamics. Overall, we show that carotenoid-based coloration is a complex and multifaceted trait. If we are to gain a better understanding of the processes responsible for the generation and maintenance of variation in carotenoid-based coloration, these complexities need to be taken into account.