Accumulating Evidence: Procedures for Resolving Contradictions among Different Research Studies

Abstract

Significant knowledge in the social sciences accrues ever too slowly. A major reason is that various research studies on a particular question tend to be of dissimilar designs, making their results difficult to compare. An even more important factor is that social science studies frequently produce conflicting results,which hinder theoretical developments and confuse those responsible for the implementation of social policies. In this pioneering effort the authors suggest criteria for determining when data from dissimilar studies can be pooled. Methods for recognizing fundamental differences in research designs, and for avoiding the creation of artificial differences, are offered. A paradigm, labeled the "cluster approach," is proposed as a means of combining the data of studies from which conflicting conclusions have been drawn. Major emphasis is placed on ways that the paradigm might solve problems presently faced by educational researchers,and several studies comparing the effectiveness of preschool programs are used to illustrate the cluster approach.