What Are We Measuring When We Evaluate Journals?

Abstract

This article undertakes two studies to examine issues related to journal rankings. Study 1 examines the consistency between journal rankings reported in past studies. It finds that while there is consistency when comparing these studies, this consistency does not always occur outside the top-ranked journals. Study 2 explores whether individuals believe that the weighting of four underlying evaluative criteria—that is, prestige, contribution to theory, contribution to practice, and contribution to teaching—vary, based on (1) whose criteria are used (individual or individuals’ perception of their institutions weighting), (2) the geographic region in which the individuals teach (North America, Europe, and Asia-Pacific), and (3) whether or not an individual works at an institution offering a Ph.D./D.B.A. The results suggest that some differences in criteria weighting exist. Implications are discussed, with it being suggested that it may not be possible to develop a universally applicable set of journal rankings.