Special Issue Editors’ Essay

Abstract

Background: Impact evaluations draw their data from two sources, namely, surveys conducted for the evaluation or administrative data collected for other purposes. Both types of data have been used to estimate program impacts. This is an introductory essay to a Special Issue entitled “Do the Estimated Effects of Social Programs Depend on the Source of Data Used to Measure Them? Survey Data Versus Administrative Data.” In addition to this essay, the Special Issue contains six articles, which appear in Volume 42, Issue 5–6 (October–December 2018) and in this issue (Volume 43, Issue 5 (October 2019)) of Evaluation Review. Objective: To describe and summarize each of the six papers and draw lessons from them. The papers investigate the relative strengths and weaknesses of survey and administrative data for estimating the impacts of policy interventions. Results: This essay first describes a simple model of the mechanisms that can cause impacts estimated with survey data to differ from those estimated with administrative data. It then describes and summarizes each of the papers appearing in this Special Issue and uses the model described to interpret the findings when it is applicable. The final section draws general lessons from the papers. Conclusions: The decision on whether to use survey or administrative data to estimate program impacts can be highly consequential because the estimates can differ considerably. All the papers in this Special Issue point to the importance of using both survey data and administrative data whenever possible.