Assessing response shifts and true changes in longitudinal evaluation research: the impact of the first COVID-19 lockdown on subjective well-being in Italy

Abstract

Latent constructs measured at multiple points in time are frequently used in social and evaluation research to assess the impact of interven-tions or relevant phenomena on individuals and groups (e.g. training programs, policies, exogenous events, etc.). However, researchers of-ten do not consider the accuracy of such models depends on the as-sumption of longitudinal measurement invariance (LMI), which states that repeatedly measured latent variables should effectively represent the same construct in the same metric over time. Non-invariance can incur when individuals recalibrate their standards of measurement, reprioritize their values or reconceptualize the target construct as a di-rect consequence of the intervention or the phenomena under study. Such response shifts may obfuscate true changes in the latent means of the construct, thus leading to inaccurate inferences. The present study aims to explore this issue by first offering a detailed description of the potential sources of response shifts and the general procedures used for LMI testing in a structural equation modelling framework. An em-pirical application is then carried out on a sample of Italian respond-ents interviewed before and immediately after the COVID-19 lock-down and, based on the results of the analysis, practical implications for evaluation research using latent outcome variables in a longitudinal setting are discussed.