We report what we believe to be the first comparative study of multi-objective genetic programming (GP) algorithms on benchmark symbolic regression and machine learning problems. We compare the Strength Pareto Evolutionary Algorithm (SPEA2), the Non-dominated Sorting Genetic Algorithm (NSGA-II) and the Pareto Converging Genetic Algorithm (PCGA) evolutionary paradigms. As well as comparing the quality of the final solutions, we also examine the speed of convergence of the three evolutionary algorithms. Based on our observations, the SPEA2-based algorithm appears to have problems controlling tree bloat—that is, the uncontrolled growth in the size of the chromosomal tree structures. The NSGA-II-based algorithm on the other hand seems to experience difficulties in locating low error solutions. Overall, the PCGA-based algorithm gives solutions with the lowest errors and the lowest mean complexity.