External Validation of Kidney Donor Risk Index Does Not Mitigate Its Basic Limitations

    loading  Checking for direct PDF access through Ovid

Excerpt

Kidney Donor Risk Index (KDRI) was developed in the United States by Rao et al1 to assess the quality of deceased donor kidneys and predict long-term graft survival. Data from 69 440 deceased donor kidneys from 1995 to 2005 were used for this analysis, and 10 donor and 4 recipient factors were identified and used in the model. The concordance statistic (C statistic) to predict graft survival for this study is modest at approximately 0.63. The United Kingdom KDRI which is based on 5 donor factors has similar C statistic of 0.62.2 Donor age, hypertension, weight, longer hospital stays before death, and use of adrenaline were found to be significant predictors for graft survival at 3 years in this model.
Because the KDRI is used to predict graft survival (without considering treatment effect), in calculations of expected outcomes and for allocation purposes, KDRI for all intents and purposes is being used as a “prognostic” biomarker. A prognostic biomarker is a marker that can predict a clinical outcome regardless of treatment.
The study by Peters-Sengers et al3 in this month's Transplantation has provided a thoughtful and well-done external validation of the original KDRI in the Dutch population. They performed validation using 3201 adult kidney recipients between years 2002 and 2012 in Netherlands. KDRIdonor-only included 9 donor factors as opposed to 10 donor factors used in original KDRI excluding ethnicity. KDRIfull included the 4 recipient factors as well.
The calibration slope for the Dutch KDRI was 0.96, indicating good correlation. The survival probabilities at 1, 3, 5, and 7.5 years for the different KDRI quintiles were comparable between the Dutch and the US cohort. Graft survival was lower at 1 year but better at 5 and 7.5 years in the Dutch population. Regression coefficients for donor age, weight, and cold ischemia time were different from the US study.
There were discrepancies in the predicted versus observed graft survival especially in the higher quintiles with KDRI of 1.45 or greater in the Dutch population, which in turn might influence donor acceptance and transplant activities.
The discriminatory ability of KDRIfull and KDRIdonor-only for graft survival at 5 years in Dutch population as measured by Harrell C statistic was 0.63 (95% confidence interval, 0.62-0.64) and 0.62 (95% confidence interval, 0.61-0.63), respectively, and was comparable to the discriminative ability for graft survival as identified in the original KDRI.4 Death-censored graft survival also had similar C statistic value. Increase in recipient age decreased the effect of KDRI on graft survival.
Minor differences aside, the Dutch study offers an excellent external validation of KDRI, and the authors should be applauded for this effort. However, a far more important and worrisome message lurks beneath the surface. Though validated, is KDRI a valid prognostic biomarker that should be used as a tool for regulatory/allocation purpose and individual patient use?
The above studies have examined and used KDRI as a prognostic biomarker for graft survival at 5 years or so. The authors report C statistic of around 0.6 which may be statistically significant but in real life has little clinical significance in predicting the clinical outcome of graft survival.
The receiver operating characteristic curve is an important tool to assess the utility of a test at predicting an outcome, which is a binary variable. It is a plot of true positive rate/sensitivity against false-positive rate/specificity of a diagnostic test.5 The utility of a test is measured using area under the receiver operating characteristic curve (AUC). The AUC corresponds to the C statistic or the ability of the test to predict outcome. C statistic of 1 implies perfect correlation between the test and outcome whereas AUC of 0.
    loading  Loading Related Articles