Methodological Challenges in Retrospective Evaluation of Mental Health Court Effectiveness

Abstract

Mental Health Courts (MHCs) are problem-solving courts that have been implemented throughout the United States. One critical component of MHCs is determining their effectiveness and limitations. However, unique challenges are encountered when evaluating MHCs. One major challenge, and the focus of this paper, is identifying an adequate control group. The ideal approach to determining efficacy is using a controlled group design whereby participants are randomized to treatment or control conditions. However, this approach is not possible when conducting retrospective evaluation of court data. In addition, a specific set of ethical and logistical issues arise. Propensity score matching (PSM) provides an alternative approach for comparing groups when randomization is not possible. PSM works by first identifying the characteristics that make a person likely to be in treatment. We describe our attempts to use PSM in a MHC evaluation. Specific challenges with PSM are discussed and recommendations are made for use of PSM with MHCs.