The Discard of Study Evidence by Literature Reviewers

Abstract

Literature reviewers have drawn dramatically different conclusions about the effectiveness of management by objectives (MBO). Some reviewers drew quite optimistic conclusions about the value of MBO; others drew pessimistic and/or highly qualified conclusions. Reviewers who drew pessimistic conclusions discarded studies that had "low quality" for one reason or another, whereas reviewers who drew optimistic conclusions were more comprehensive in approach. The studies that were discarded by reviewers were found to be run in contexts in which MBO was much more likely to succeed. The discard of study evidence because reviewers believed it had "low quality" was in this case a prescription for error Indeed, in most research domains, hypotheses about which study features constitute "low quality" should probably be regarded as false.