Response to Letter to the Editor Regarding, Dissecting Leapfrog: How Well Do Leapfrog Safe Practices Score Correlate With Hospital Compare Ratings and Penalties, and How Much Do They Matter?
We thank Ms Binder and The Leapfrog Group for the response and description of the extensive verification procedures applied to data submitted to the Leapfrog Hospital Survey. We recognize the efforts of The Leapfrog Group in advancing and innovating complex methods for evaluating and comparing hospital safety, and are grateful for their commitment to providing access to their data and transparent methodologies. As researchers invested in improving health care quality and safety, we wish to clarify that our motivation for these analyses1 was indeed to better understand and describe the application of the methodology to generate the publicly reported Leapfrog Hospital Safety Grades, and to help inform future refinements if felt appropriate. It was not our intent to make suggestions to hospitals as to how to impact their grade.
We also fully recognize and commend the complexity of the Leapfrog Hospital Safety Grade, incorporating multiple process and outcome measures. Indeed, this complexity is what piqued our interest to study the methodology’s application, and investigate the association between Leapfrog’s Safe Practices process measures with publicly reported outcomes and penalties (objective 1). As this is our first analysis involving Leapfrog data, we were actually surprised to learn that hospitals receive a Leapfrog Hospital Safety Grade whether or not they submit the voluntary Leapfrog Hospital Survey, which includes the Safe Practices measures. This prompted our curiosity in how the various grade components contribute to the grade for hospitals that report compared with hospitals that choose not to report to Leapfrog (objective 2).
Ms Binder raises an important concern that misalignment across reporting periods and other factors may be responsible for our ancillary finding of reported differences between primary and secondary data sources for standardized infection ratios (SIRS) for central line-associated bloodstream infections (CLABSI) and catheter-associated urinary tract infections (CAUTI). To clarify the procedures used for aligning data for our analyses involving the CAUTI and CLABSI SIR outcomes, we have provided a Supplemental Appendix (Supplemental Digital Content 1, http://links.lww.com/MLR/B381). We hope that sharing this document will be useful for The Leapfrog Group and for future researchers working with Leapfrog data.
To summarize these steps, The Leapfrog Methodology Report2 for the Spring 2014 Hospital Safety Score indicates 2 data sources were used for CLABSI and CAUTI SIR outcomes. The 2013 Leapfrog Hospital Survey, with a data collection period from January 1, 2012 to June 30, 2013, was the primary data source for hospitals that chose to participate. The secondary data source was Centers for Medicare and Medicaid Services (CMS)’s Hospital Compare, with a data collection period from April 1, 2012 to March 31, 2013,3 and was used when hospitals did not report to Leapfrog or did not meet Leapfrog’s minimum reporting requirements for these measures. The Leapfrog Hospital Safety Grade dataset includes the CAUTI and CLABSI SIR used for each hospital, as well as an indicator for the source of the data (Leapfrog Hospital Survey or Hospital Compare). As our analyses used CMS Hospital Compare for all Leapfrog hospitals, regardless of Hospital Survey completion, we merged these data with the Hospital Safety Grade dataset. To confirm we had pulled the correct Hospital Compare dataset, we examined the correlation between CAUTI and CLABSI SIRs included from hospitals identified in the Leapfrog dataset as having CMS data with our merged Hospital Compare data, and (after accounting for Leapfrog’s trimming) found a perfect correlation (r=1.00) (Supplemental Appendix Figures A and B, Supplemental Digital Content 1, http://links.lww.com/MLR/B381). We then compared the distribution of CLABSI and CAUTI SIRs across data sources for those hospitals identified as having the Leapfrog Hospital Survey as their data source.