Does Researcher Allegiance Bias Outcomes in Psychotherapy Research? A Quasi‐Experimental Secondary Analysis

Abstract

ABSTRACTResearchers who conduct studies comparing the efficacy of two treatments often find that their preferred treatment outperforms the comparison treatment. This finding has been labelled the allegiance association. Although this association is robust, it is unclear whether it reflects an allegiance bias on the part of the researchers or whether it is noncausal, with researchers being allied to the more effective treatments. This study applied a quasi‐experimental method proposed by a previous study to 19 pairs of treatment comparison studies. Each member of a pair had used the same two psychotherapies to treat clients with the same disorder, but the researchers in each of the two studies had opposing allegiances. If the authors of one study in the pair concluded that their preferred treatment was superior and the authors of the other study concluded that their preferred treatment was superior or that the two treatments were equivalent, these patterns would suggest allegiance bias. In 10 of the 19 pairs, the patterns were consistent with the operation of an allegiance bias, indicating that although allegiance biases are not inevitable, they are ubiquitous. Practitioners and other psychotherapy research consumers should use caution when interpreting the findings from treatment comparison studies.