Self-incompatibility (SI) is an important plant reproduction mechanism that facilitates the maintenance of genetic diversity within species. Three plant families, the Solanaceae, Rosaceae and Plantaginaceae, share an S-RNase-based gametophytic SI (GSI) system that involves a single S-RNase as the pistil S determinant and several F-box genes as pollen S determinants that act via non-self-recognition. Previous evidence has suggested a specific self-recognition mechanism in Prunus (Rosaceae), raising questions about the generality of the S-RNase-based GSI system. We investigated the evolution of the pollen S determinant by comparing the sequences of the Prunus S haplotype-specific F-box gene (SFB) with those of its orthologs in other angiosperm genomes. Our results indicate that the Prunus SFB does not cluster with the pollen S of other plants and diverged early after the establishment of the Eudicots. Our results further indicate multiple F-box gene duplication events, specifically in the Rosaceae family, and suggest that the Prunus SFB gene originated in a recent Prunus-specific gene duplication event. Transcriptomic and evolutionary analyses of the Prunus S paralogs are consistent with the establishment of a Prunus-specific SI system, and the possibility of subfunctionalization differentiating the newly generated SFB from the original pollen S determinant.