A close-form expression for the exact Pair-wise Error Probability (PEP) of Space-Time (S-T) codes in Rayleigh fading channel is derived using the general and close-form solution for the probability-density function (PDF) of a sum of independent exponential distributed random variables. The expression requires evaluating the coefficients for partial fraction expansion, so an easy analytical way is proposed for doing this. The exact PEP is subsequently used to develop a simple PEP using the upper bound. Both PEPs are used in the Union bound for error rate evaluation. Numerical calculations and Monte Carlo computer simulation are used to study the accuracies of these Union bounds for error rate evaluation of a rotation-based diagonal S-T code (D code) in Rayleigh fading channels. Four other PEPs based on different bounds, i.e., the Chernoff bound, the asymptotic bound, the tight asymptotic bound, and the Eigen-Geometric-Mean (EGM) bound, are also studied for comparison. Results show that our derived close-form PEP is an exact PEP and our proposed PEP is a very tight bound to the exact PEP.