Do Research Literatures Give Correct Answers?

Abstract

The rationale for insisting on properly designed studies is to ensure the construction of research literatures that are not influenced by biases of any sort. Once a literature is established, however, whatever answers it gives (usually via meta-analysis) are accepted as valid. The results of our favored statistical techniques are understood best when significant relationships are present while the meaning of null results are unclear. Thus, significant findings are more likely to be published. This file drawer problem has been known for many years. However, the effect that the file drawer has on research literatures has never been probed empirically. A new method is described that tests the impact a file drawer has on a research literature. The results of the file drawer on four separate psychological research literatures are presented.