Despite little evidence of effectiveness, pay-for-performance programs are being adopted with the intent of improving the quality of care. The few studies evaluating these programs have shown only modest and short-term effects on hospital processes of care and even weaker evidence for effects on patient outcomes. In 2008, the Advancing Quality program was introduced in all 24 National Health Service hospitals in the northwest region of England that provided emergency care. Patient-level data were used to analyze patient mortality from all hospitals across England for 3 conditions included in the program and 6 conditions not included in the program for 18 months before and 18 months after the introduction of the program.
The Advancing Quality program was the first hospital-based pay-for-performance program to be introduced in England. Hospitals were required to collect and submit data on 28 quality measures covering acute myocardial infarction, coronary artery bypass grafting, heart failure, hip and knee surgery, and pneumonia. At the end of the first year, hospitals that reported quality scores in the top and second quartiles received bonuses. For the next 6 months, the reward system changed so that bonuses could be earned on the basis of 3 criteria, attainment, improvement, and achievement, with their performance during the second year compared with the first year. Thereafter, the program was included in a pay-for-performance program that applied across all of England and involved withholding of payments rather than bonuses. Patient-level data were obtained for all patients treated for 1 of 3 conditions included in the program: acute myocardial infarction, heart failure, and pneumonia. All deaths that occurred within 30 days after admission were included in the analysis. Equivalent data were obtained for patients admitted for 6 diagnoses not included in the program (acute renal failure, alcoholic liver disease, intracranial injury, paralytic ileus and intestinal obstruction without hernia, pulmonary embolism, and duodenal ulcer). The 3-year period (2007–2010) included 18 months before the program’s introduction and the first 18 months of its operation. The final sample included 410,384 patients with pneumonia, 201,003 patients with heart failure, 245,187 patients with acute myocardial infarction, and 241,009 patients with conditions not included in the program. Between-region and within-region difference-in-differences analyses were used to compare changes in mortality over time between the northwest region and the rest of England for conditions included or not included in the program. Each analysis included an interaction term between the intervention group and the period after the implementation.
For all conditions, patients in the northwest region were slightly younger but had more coexisting conditions. Similar changes over time in patient volumes and patient characteristics were observed in both areas. Risk-adjusted mortality for all the conditions decreased over the 3 years in the northwest region and the rest of England. The reduction in mortality for conditions included in the program was 21.9% to 20.1% in the northwest region compared with 20.2% to 19.3% in the rest of the country. As compared with overall mortality for conditions not included in the program within the northwest region (within-region difference-in-differences analysis), a significantly greater reduction (0.9 percentage points) in overall mortality was seen for conditions included in the program, with a significant reduction only for pneumonia. The between-region difference-in-differences analysis found a significantly greater reduction of 0.9 percentage points in overall mortality in the northwest region, with individually significant reductions only for pneumonia. Combining these 2 methods in the triple-difference analysis suggested a greater overall reduction in mortality of 1.3 percentage points in the northwest region. This is a relative rate reduction of 6% and, during the 18-month period, equates to a reduction of 890 deaths in the total population of 70,644 patients with these conditions in the northwest region of England. The reduction in mortality for conditions not included in the program did not differ significantly between the 2 geographical areas. Risk-adjusted mortality for the conditions not included in the program decreased by similar amounts in the studied regions. No significant changes in the proportion of patients discharged to care institutions were noted, and all differences were smaller than 0.3 percentage points.
In response to the program, participating hospitals adopted a range of quality improvement strategies. Although this program was similar to an initiative in the United States, the different results indicate that the context and implementation of such incentive programs have a close association with their outcomes. The possibility exists that incentives can have an impact on mortality.