Prostate-Specific Antigen (PSA) is a crucial biomarker for screening prostate cancer, but a sensitive and selective immunosensor for rapid quantification of serum PSA remains to be developed. In this study, a sandwich pair of nanobodies (Nbs) (i.e., Nb2 and Nb40) against PSA surface antigen was obtained from an alpaca-derived immune phage display library. A sandwich-type immunosensor for the sensitive and selective detection of PSA in serum samples was ingeniously designed based on the pair of Nbs. The small size of Nb40 allowed high capture densities on the surface of reduced graphene oxide (rGO) nanocomposed with massive Au nanoparticles (rGO@AuNPs), which significantly improved the conductivity and provided a large area to anchor many primary antibodies. The secondary antibody Nb2 fused with streptavidin -binding peptide (SBP) cooperated with Nb40 for PSA sandwiching. Accompanying introduction of horseradish peroxidase-streptavidin (HRP-SA) coupled with Nb2-SBP, the faradaic current was linearly correlated with the logarithm of PSA concentration in a range of 0.1–100ng mL−1. More importantly, this immunosensor exhibited excellent selectivity, stability, and reproducibility due to the sandwich pair Nbs. The proposed immunosensor was successfully applied in determining PSA in serum samples and could be used for the sensitive and specific detection of PSA.