Recent advances have led to a steady improvement in cancer treatments. The increasing number of therapeutic options and the corresponding improvement in outcomes pose a number of challenges for comparative effectiveness research in oncology. This review is focused on the choice of primary end points and their interpretation in randomized clinical trials that are designed to inform patients and clinicians on the relative benefits of cancer therapies. We discuss end points that directly measure clinical benefit as well as end points that are thought to be surrogates for clinical benefit. Particular attention is given to the issues associated with the use of overall survival as the primary end point in randomized clinical trials.