The inscrutable baseline and the problem of attribution

Abstract

AbstractThis paper examines the problem of attribution in the evaluation of energy efficiency program impact. The methodological problem concerns the observability of consumer behavior under the baseline condition of no program intervention. The statistical solution to the problem, which entails randomized exposure of targeted individuals to program influence, is not a viable alternative in most applications. Randomized opt-in and randomized encouragement designs do not conform to this requirement because all targeted individuals are encouraged to participate in the program, resulting in negative exposure bias. Quasi-experimental methods which utilize non-targeted individuals or targeted nonparticipants as baseline surrogates are further subject to selection bias of unknown magnitude and direction. Valid attribution in the general case of unrestricted eligibility depends on prior knowledge of the determinants of measure adoption and program participation. In default of such knowledge, evaluators must rely upon structural assumptions that have no foundation in empirical science. On the other hand, established measurement and verification methods which exploit scientific knowledge of the determinants of end-use energy consumption should be utilized to obtain unbiased estimates of individual measure and gross program energy savings.