Objectives: Pre-procedure risk assessment is central to clinical decision making for patients with advanced valvular heart disease (VHD) and treatments are increasingly being offered to patients with elevated pre-procedure risk. While there are numerous clinical predictive models (CPMs) available for patients with VHD, the relative performance of these CPMs is largely unknown. Here we describe the performance of CPMs available for patients with VHD with specific attention to whether CPMs have been externally validated.
Methods: To identify CPMs for patients with VHD, we conducted a systematic review of the Tufts PACE CPM Registry, a comprehensive database of cardiovascular CPMs. For each identified CPM for patients with VHD, we performed a complete citation search using Scopus to identify any external validations of these models published in other articles. We extracted information on CPM performance in both the original report and also the external validations. For external validations we calculated the relative percent decrease in discrimination.
Results: We identified 41 CPMs predicting outcomes for patients with VHD. 33 (81%) predict outcomes following surgical intervention, 5 (12%) predict outcomes following percutaneous interventions, and 3 (7%) predict outcomes in the absence of intervention. Only 30/41 (73%) of the CPMs report a c-statistic. The median reported c-statistic was 0.77 [IQR, 0.04] for CPMs predicting outcomes following surgical interventions, 0.68 [IQR, 0.04] for CPMs for percutaneous interventions, and 0.83 [IQR, 0.07] for CPMs predicting outcomes in the absence of intervention. While a total of 69 external validations of these CPMs have been published, only 21 (51%) of the CPMs have ever been externally validated. For external validations that report c-statistics, we noted a median percent decrement in discrimination of -27.6% [IQR, -37.4] (Figure).
Conclusion: While there are numerous CPMs for patients with VHD, performance is often incompletely reported and half of these CPMs have never been externally validated. The CPMs that have been externally validated generally show substantially worse discrimination in external datasets compared to the derivation datasets.