Welfare Reform and Addiction: A Priori Hypotheses, Post Hoc Explorations, and Assisted Sensemaking in Evaluating the Effects of Terminating Benefits for Chronic Substance Abusers

Abstract

The passage of the Contract with America Advancement Act terminated the Social Security Administration’s Disability Insurance and Supplemental Security Income benefits for persons diagnosed with drug or alcohol addiction (also known as DA&A recipients), effective January 1, 1997. From an evaluation standpoint, the law created the opportunity for a “natural” experiment, specifically, an interrupted time series quasi-experiment. Through the availability of person-level data on an entire state population, the present study combines the design tradition of interrupted time series with the analytic tradition of growth curve modeling to create an “enriched” interrupted time series with greater flexibility and explanatory power than the standard design based on aggregate data. Additional techniques from the contemporary evaluation toolkit include: use of propensity scoring for matching and covariance adjustment, multilevel modeling, alternate estimates of the counter-factual, and sensitivity analyses to empirically test the plausibility of validity threats. The study assesses the net impact of the law on labor force participation and criminal behavior among former DA&A recipients in Washington State, using person-level data on employment and arrests from 2 years before implementation to 3 years after. Hypothesis tests showed unambiguously that the law increased employment but did not increase arrests. Post hoc exploratory analyses—framed by Mark et al. [Mark, M. M., Henry, G. T., & Julnes, G. (2000). Evaluation: An integrated framework for understanding, guiding, and improving policies and programs. San Francisco: Jossey Bass] theory of evaluation as assisted sensemaking with particular attention to generative mechanisms and social betterment—suggest a more nuanced interpretation. The combination of a priori and post hoc analyses confirmed that, as a result of the law, there were fewer DA&As on public supports, more DA&As in the workforce, and no evidence of more criminals menacing the public. Analyses also confirmed that a segment of the population, already poor, was made destitute.