Considerations for Evaluating Treatment Effects From Randomized Clinical Trials

    loading  Checking for direct PDF access through Ovid


In drug development, randomized clinical trials (RCTs) are considered the gold standard for trial design. In such trials, patients are assigned to study treatments randomly. Randomization results in comparable groups in terms of patient characteristics at baseline, such that any difference observed after randomization is likely to be due to the assigned treatments. Unfortunately, randomization does not protect from confounding and bias due to events that occur after randomization; for example, variable adherence to the prescribed regimen, discontinuation of treatment, treatment switching, etc. Such events by themselves provide information about the treatment effects; for example, the treatment may cause patients to discontinue from the study due to adverse events. Traditionally, an intention‐to‐treat (ITT) analysis, also considered by many as the gold standard for statistical analysis of an RCT, is conducted. The ITT approach came to prominence decades ago1 and was institutionalized in drug development with the publication of the ICH‐E92 guideline, which states the following:
The resulting ITT analysis yields a treatment effect estimate that is often described as the effect of the treatment assigned at randomization or the treatment policy effect. Despite its ubiquitous use in randomized, controlled clinical trials, the ITT analysis and its interpretation are neither consistently applied3 nor well understood by many.4 As indicated in the ICH‐E9 guideline, an ITT analysis generally includes all data on all randomized patients, regardless of adherence to the randomized study treatments. The resulting estimates are often difficult to interpret and may not provide an intuitive or clinically meaningful estimate of treatment effects.4 Nonetheless, regulatory agencies often require ITT analyses for pivotal registration trials and make decisions based on such. Paradoxically, their subsequent marketing authorization and approved label are for specific treatments rather than treatment policies.
For a simple illustration, let us consider a 26‐week study, using 1:1 randomization of patients with uncontrolled diabetes, for a novel glucose‐lowering treatment (T) or a control treatment (C). For simplicity of calculations, let us suppose 50% of the control patients discontinue treatment due to lack of efficacy (no change in glycated hemoglobin (HbA1c) values) and 50% persist to the end with no average change in HbA1c. Further, of the patients on the novel treatment, 50% discontinue treatment due to tolerability issues with no average change in HbA1c, and 50% complete the study with acceptable tolerability and an average 2% reduction in HbA1c. A schematic illustration of this example is shown in Figure1.
An ITT analysis of this hypothetical trial includes all randomized patients, where the estimate of T vs. C would be as follows: JOURNAL/cpth/04.02/00003098-201712000-00016/math_16MM1/v/2017-11-12T175837Z/r/image-png
In diabetes trials such a difference is generally regarded as clinically meaningful, and with a large enough sample size, such a difference could be declared statistically significant. However, regardless of statistical significance, how should one interpret this finding? Would physicians tell their patients to expect an HbA1c reduction of 1% or 2%, keeping in mind that perhaps no one in the trial experienced a 1% reduction? Would payers make coverage decisions based on the overall average reduction of 1% or based on a reduction of 2% in patients who are able to take the medication for the intended duration? Would regulators approve a drug label with the estimate of a 1% or a 2% reduction in HbA1c? This is further complicated if patients are allowed to take rescue medication (R) in case of a lack of acceptable efficacy. In this situation, the ITT analysis estimates a “treatment policy” effect (T+R vs. C+R) rather than a “direct treatment” effect (T vs. C).
    loading  Loading Related Articles