An Open Source Replication of a Winning Recidivism Prediction Model

Abstract

We present results of our winning solution to the National Institute of Justice recidivism forecasting challenge. Our team, “MCHawks,” placed highly in both terms of accuracy (as measured via the Brier score), as well as the fairness criteria (weighted by differences in false positive rates between White and Black parolees). We used a non-linear machine learning model, XGBoost, although we detail our search of different model specifications, as many different models’ predictive performance is very similar. Our solution to balancing false positive rates is trivial; we bias predictions to always be “low risk” so false positive rates for each racial group are zero. We discuss changes to the fairness metric to promote non-trivial solutions. By providing open-source replication materials, it is within the capabilities of others to build just as accurate models without extensive statistical expertise or computational resources.