Implementing Clinical Prediction Models: Pushing the Needle Towards Precision Pharmacotherapy
Response heterogeneity remains a reality of clinical practice. Even where highly efficacious drug therapy prevails, many patients do not respond. Prediction models may serve as a useful means of addressing response heterogeneity. The process of generating a prediction model typically involves integrating several pieces of information to generate a response prediction. The process begins with data acquisition, cleaning and processing before model derivation and validation may be performed. While these have been discussed in the literature at length, considerably less guidance is available on how such models may be brought to practice to actualize precision drug therapy.2
The critical aspect of model validation that remains unanswered is that of its adequacy in warranting model implementation. Specifically, to what extent does retrospective validation provide evidence of model safety and efficacy? Could excellent retrospective validation performance eliminate the need to evaluate models in a randomized prospective trial? While necessary, retrospective validation alone is insufficient to guarantee improvements in clinical outcomes following implementation. Several other factors should be considered when determining the sufficiency of evidence warranting clinical implementation (Table1). Our proposed list includes considerations relating to the specific clinical problem and factors related to the model itself.
While randomized trials can reveal safety and efficacy, there may be genuine circumstances where bypassing a trial is ethically justifiable (Figure1). Prospective trials are costly and slow to complete. Accelerated model implementation may be indicated when there is genuine urgency for alternative therapeutic approaches. Nonetheless, such decisions should be arrived at after a thorough evaluation of the evidence and clinical circumstances. Unfortunately, clear guidelines are lacking to facilitate ethical implementation of prediction models, considering the vast number of prediction models that have been constructed and validated but unimplemented in clinical practice.
Productizing models in a suitable form for clinical use is a vital consideration in ensuring their accurate and sustained use. Traditionally, prediction models have manifested as pen‐and‐paper scoring tables and/or nomograms to facilitate point‐of‐care predictions.3 Arguably, these still remain preferred but entail disadvantages, including the need to categorize continuous predictors and to round off coefficients for quick and easy risk assessment. However, these procedures can blunt a model's accuracy. With the availability of computational resources at the point‐of‐care, these limitations may be avoided. Models can now be implemented as mobile/web applications or even be integrated with electronic health records for passively generating predictions.4 Computationally productized models may also facilitate risk interpretation through model‐based visualizations, instantaneously generated and displayed at the point‐of‐care. Such individualized risk profiles may facilitate risk communication, creating the opportunity for shared decision‐making using objective evidence (e.g., http://bit.ly/Stage4PrognosticScore). Given the myriad of possibilities, model developers should work closely with clinicians to develop tools that suit enduser needs and preferences.
Following productization, rigorous, prospective trial‐based evidence demonstrating safety and efficacy of prediction models would serve as a strong justification for clinical implementation.