|| Checking for direct PDF access through Ovid
The National Quality Forum has endorsed the use of a modified version of the Mortality Probability Model (MPM0-III) for comparing observed and predicted hospital mortality in U.S. ICUs. However, recent studies suggest that this model is less accurate than current versions of other prognostic models.That the performance of MPM0-III and APACHE IV for predicting hospital mortality would not differ when tested concurrently in a large multi-institutional ICU database.Retrospective cohort study using day 1 information from 51,825 first admissions in 46 ICUs at 34 U.S. hospitals during 1/1/2008 to 6/30/2012. For each MPM0-III-eligible patient we calculated the probability of hospital mortality using the MPM0-III and APACHE IV predictive models. We compared each model’s prediction against actual mortality using the following measures of accuracy: discrimination was assessed by the area under the receiver operating characteristic curve (AUROC); calibration was assessed by the standardized mortality ratio (SMR), Hosmer-Lemeshow (H-L) statistic, and a modified Brier score. The latter measure consisted of determining the Brier score resulting from using the observed mortality as a constant prediction, and then calculating the percentage reduction the actual Brier score represented (higher percentage = better accuracy).The observed hospital mortality rate was 11.2%, while the APACHE IV predicted mortality was 12.6% (SMR = 0.89), and MPM0-III predicted mortality was 15.2% (SMR = 0.74). For APACHE IV the AUROC was 0.884 and H-L statistic was 12.4. For MPM0-III the AUROC was 0.838 and H-L statistic 109.8. Differences between the two models for the AUROC and SMR, respectively were highly significant (p<0.001). The percent reduction in prediction error from the null model shown by the Brier score was 31.4% for APACHE IV and 18.5% for MPM0-III.In a side-by-side comparison within a large dataset, the APACHE IV prognostic model had better discrimination and calibration than the MPM0-III model. Healthcare reporting agencies should take this information into account when considering critical care outcome measures.