Imperfect Tools: A Research Note on Developing, Applying, and Increasing Understanding of Criminal Justice Risk Assessments

Abstract

This article shares considerations for designing, implementing, and understanding risk assessments used to reduce recidivism of people under community supervision. These insights are gleaned from 27 data scientists who participated in focus groups during the National Institute of Justice’s Recidivism Challenge Winners Symposium. Analyses revealed three primary themes: design considerations, implementation, and increasing awareness and understanding of risk assessments. Critical aspects of the design phase include validating the tool, incorporating field data that account for real-time changes, and adopting strategies to address false positives/negatives and the model’s complexity. Upon the tool’s development, practitioners are recommended to devise an implementation plan, balance attention to risk with client-focused needs, and exercise modest discretion while considering algorithmic results. Recognizing the value predictive instruments bring to decision-making and identifying their limitations is needed to increase understanding for all stakeholders. Collaboration and dialogue between tool developers and practitioners are crucial at every stage.