What to Do When Everything Happens at Once: Analytic Approaches to Estimate the Health Effects of Co-Occurring Social Policies

Abstract

Abstract Social policies have great potential to improve population health and reduce health disparities. Increasingly, those doing empirical research have sought to quantify the health effects of social policies by exploiting variation in the timing of policy changes across places. Multiple social policies are often adopted simultaneously or in close succession in the same locations, creating co-occurrence that must be handled analytically for valid inferences. Although this is a substantial methodological challenge for researchers aiming to isolate social policy effects, only in a limited number of studies have researchers systematically considered analytic solutions within a causal framework or assessed whether these solutions are being adopted. We designated 7 analytic solutions to policy co-occurrence, including efforts to disentangle individual policy effects and efforts to estimate the combined effects of co-occurring policies. We used an existing systematic review of social policies and health to evaluate how often policy co-occurrence is identified as a threat to validity and how often each analytic solution is applied in practice. Of the 55 studies, only in 17 (31%) did authors report checking for any co-occurring policies, although in 36 studies (67%), at least 1 approach was used that helps address policy co-occurrence. The most common approaches were adjusting for measures of co-occurring policies; defining the outcome on subpopulations likely to be affected by the policy of interest (but not other co-occurring policies); and selecting a less-correlated measure of policy exposure. As health research increasingly focuses on policy changes, we must systematically assess policy co-occurrence and apply analytic solutions to strengthen studies on the health effects of social policies.