Methodological Uncertainty and Multi-Strategy Analysis: Case Study of the Long-Term Effects of Government Sponsored Youth Training on Occupational Mobility

Abstract

Sociological practitioners often face considerable methodological uncertainty when undertaking a quantitative analysis. This methodological uncertainty encompasses both data construction (e.g. defining variables) and analysis (e.g. selecting and specifying a modelling procedure). Methodological uncertainty can lead to results that are fragile and arbitrary. Yet, many practitioners may be unaware of the potential scale of methodological uncertainty in quantitative analysis, and the recent emergence of techniques for addressing it. Recent proposals for ‘multi-strategy’ approaches seek to identify and manage methodological uncertainty in quantitative analysis. We present a case-study of a multi-strategy analysis, applied to the problem of estimating the long-term impact of 1980s UK government-sponsored youth training. We use this case study to further highlight the problem of cumulative methodological fragilities in applied quantitative sociology and to discuss and help develop multi-strategy analysis as a tool to address them.