Policy experiments: Investigating effectiveness or confirming direction?

Abstract

In England, ‘policy experiments’ are largely synonymous with the use of randomized controlled trials (RCTs) to test whether one policy ‘works’ better than another. While advocacy of the use of RCTs in public policy presents this as relatively straightforward, even common sense, the reality is different, as shown through analysis of three high profile policy pilots and their evaluations undertaken in health and social care in England in the mid/late 2000s. The RCTs were expected to confirm the direction of policy by resolving any remaining uncertainty about the effectiveness of the chosen path and their existence was used largely as instruments of persuasion. The findings from the analysis of the three pilots confirm the continuing relevance of Campbell’s 1969 insight that governments struggle to experiment in the scientific sense and explain the limited effect of these policy experiments on policy decisions.