Avoiding Pitfalls in Applying Prediction Models, As Illustrated by the Example of Prostate Cancer Diagnosis

Abstract

BACKGROUND The use of different mathematical models to support medical decisions is accompanied by increasing uncertainties when they are applied in practice. Using prostate cancer (PCa) risk models as an example, we recommend requirements for model development and draw attention to possible pitfalls so as to avoid the uncritical use of these models. CONTENT We conducted MEDLINE searches for applications of multivariate models supporting the prediction of PCa risk. We critically reviewed the methodological aspects of model development and the biological and analytical variability of the parameters used for model development. In addition, we reviewed the role of prostate biopsy as the gold standard for confirming diagnoses. In addition, we analyzed different methods of model evaluation with respect to their application to different populations. When using models in clinical practice, one must validate the results with a population from the application field. Typical model characteristics (such as discrimination performance and calibration) and methods for assessing the risk of a decision should be used when evaluating a model's output. The choice of a model should be based on these results and on the practicality of its use. SUMMARY To avoid possible errors in applying prediction models (the risk of PCa, for example) requires examining the possible pitfalls of the underlying mathematical models in the context of the individual case. The main tools for this purpose are discrimination, calibration, and decision curve analysis.