Reply to Letter: “Redesigning ACS-NSQIP Data Collection and Reports Will This Translate Into Better Outcomes?”
We appreciate the thoughtful editorial by Osborne and Goodney1 about our 4 articles on the Surgical Risk Preoperative Assessment System (SURPAS) recently published in Annals of Surgery.2–5 We would like to make comments on some of their points.
We were somewhat perplexed with the title of the editorial, “Redesigning ACS-NSQIP Data Collection and Reports Will This Translate into Better Outcomes?” We do not intend to influence the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) data collection process and its use in more than 600 hospitals to make operative outcomes comparison credible by risk-adjustment. At the University of Colorado, we view the ACS-NSQIP and SURPAS as separate surgical quality improvement efforts that complement one another. The University of Colorado (UC) Hospital joined the ACS-NSQIP so that we could compare our risk-adjusted adverse surgical outcomes with other surgical programs around the country. When UC Hospital implements SURPAS in the future, it will be to provide patient risk assessments preoperatively to counsel patients and their families and to feedback these risk assessments to the surgical teams so that they might have the opportunity to mitigate risk in moderate and high risk patients before, during, and after surgery, with the ultimate goal of reducing postoperative adverse events in these patients.
We agree with the editorial that the 4 articles do not address the issue of implementation and use of SURPAS. These 4 articles present some of the statistical foundation of the system that we have been working on over the past 3 years using the very valuable ACS-NSQIP Participant Use File. The SURPAS system will probably represent a 5 to 10 year research effort involving the development of the parsimonious statistical prediction models; obtaining feedback from surgeons, patients, and administrators; incorporating the shared decision making tool into the electronic health record system at the UC Hospital; incorporating suggestions for care management to reduce the risk in moderate and high risk patients; and implementing the tool on a pilot basis in some surgical clinics. If these steps are successful, we plan to apply the system more broadly at the UC Hospital and perhaps other hospitals in the UC Health system and then test to see whether the SURPAS system will reduce postoperative adverse events. Future progress on the SURPAS system will be reported in future articles.
We also agree with the editorial comment that the statistical technique of factor analysis may produce unintended consequences in reducing the 18 ACS-NSQIP postoperative complications to a smaller number of clusters of complications. The identification of adverse postoperative outcomes to target for quality improvement should include a combination of statistical and clinical reasoning. The SURPAS I article3 resulted in some clusters of complications (eg, combining myocardial infarction, cardiac arrest, and postoperative bleeding requiring transfusion) that involve different processes and structures of care, and, therefore, should possibly be considered separately. As SURPAS evolves, we anticipate using both clinical and statistical reasoning to identify and define those adverse postoperative outcomes that will be targeted for intervention.
In developing SURPAS, we could have taken the approach of developing separate prediction models for each type of major operation performed at the UC Hospital. This would have involved hundreds of different prediction models, with each probably requiring a different set of predictor variables. We believe this approach would lead to a cumbersome system, which would never be used by providers.