Methods for Quantifying the Clinical Significance of Change During Intervention Program Participation

Abstract

Assessing the practical or clinical significance (CS) of an intervention program’s outcomes is useful in determining its effectiveness. The CS approach gives information beyond traditional analyses by quantifying the proportions of people who meaningfully improve and deteriorate. We link latent transition analyses (LTA) to the CS literature and use a case study to contrast it with the long-standing Jacobson and Truax (JT) approach. Data came from 2,717 individuals convicted of a substance-related offense who participated in an indicated prevention program Prime For Life® (PFL). We selected outcomes describing drinking beliefs and behavior. Both CS approaches categorized a majority of participants as improved (i.e., transitioning from baseline subgroups with risky behaviors and cognitions into posttest subgroups showing lower risk). Results demonstrate how the JT approach allows the assessment of improvements on individual outcomes, while the LTA provides more nuanced information about risk groupings. Selecting a CS approach depends on research goals, availability of normative data, and data considerations. JT is an appropriate method when evaluating single outcomes. In contrast, LTA is better when a multivariate description is desired, advanced missing data handling methods are needed, or outcomes are not normally distributed. Although infrequently done, evaluating CS provides useful information about program effectiveness.