Comment on Willig’s “A Meta-Analysis of Selected Studies in the Effectiveness of Bilingual Education”

Abstract

A meta-analysis of a subset of the bilingual education evaluation literature failed in its attempt to compare meta-analysis and narrative review because the meta-analysis (a) included only one third of the studies in the narrative review, (b) addressed different questions than did the narrative review, and (c) redefined a study demonstrating the failure of bilingual education as a success. Moreover, the meta-analysis is not an appropriate analytic technique for this literature because (a) inappropriate statistics were applied to the data, (b) the bilingual education treatment was confounded with other treatments in several of the studies reviewed, and (c) the studies reviewed were not sufficiently homogeneous to support a meta-analysis. Finally, the author’s conclusion that bilingual education is effective was not supported because (a) the results of the analysis do not generalize, (b) inappropriate statistical analysis was used to arrive at this conclusion, and (c) the effects of confounded treatments were not considered.