A review and analysis of patterns of design decisions in recent media effects research

Abstract

This essay presents a critical analysis of patterns of research design decisions exhibited by authors of recently published empirical tests of media effects. The content of 211 articles published from 2010 to 2015 in six core communication journals were analyzed to document the design decisions made by the authors concerning their use of theory, sampling, measurement, and experiments. We also recorded the amount of variance explained by their tests and use this indicator of strength of findings to explain the patterns of methodological design decisions. The findings indicate that authors of these studies commonly select weaker design options over stronger ones. The reasons for these patterns are explored then critiqued leading to a series of recommendations calling for an evolution in thinking in the areas of method, theory, and paradigm. The methods recommendations attempt to increase (a) awareness of the advantages and disadvantages of options available for each design decision, (b) an understanding that often assumptions made to justify the selection of an option are faulty, and (c) a commitment to meeting a higher degree of challenges. The theory recommendations focus on increasing an understanding about why designers of most tests of media effects ignore the many theories available when designing their studies. And the paradigm recommendations focus on examining more critically the assumptions we make about the nature of human beings, the purpose of our research as challenges evolve, and the defaults in practices we have established in an exploratory phase.