MSE FINDR: A Shiny R Application to Estimate Mean Square Error Using Treatment Means and Post Hoc Test Results

Abstract

Research synthesis methods such as meta-analysis rely primarily on appropriate summary statistics (i.e., means and variance) of a response of interest for implementation to draw general conclusions from a body of research. A commonly encountered problem arises when a measure of variability of a response across a study is not explicitly provided in the summary statistics of primary studies. Typically, these otherwise credible studies are omitted in research synthesis, leading to potential small-study effects and loss of statistical power. We present MSE FINDR, a user-friendly Shiny R application for estimating the mean square error (i.e., within-study residual variance, [Formula: see text]) for continuous outcomes from analysis of variance (ANOVA)-type studies, with specific experimental designs and treatment structures (Latin square, completely randomized, randomized complete block, two-way factorial, and split-plot designs). MSE FINDR accomplishes this by using commonly reported information on treatment means, significance level (α), number of replicates, and post hoc mean separation tests (Fisher’s least significant difference [LSD], Tukey’s honest significant difference [HSD], Bonferroni, Šidák, and Scheffé). Users upload a CSV file containing the relevant information reported in the study and specify the experimental design and post hoc test that was applied in the analysis of the underlying data. MSE FINDR then proceeds to recover [Formula: see text] based on user-provided study information. The recovered within-study variance can be downloaded and exported as a CSV file. Simulations of trials with a variable number of treatments and treatment effects showed that the MSE FINDR-recovered [Formula: see text] was an accurate predictor of the actual ANOVA [Formula: see text] for one-way experimental designs when summary statistics (i.e., means, variance, and post hoc results) were available for the single factor. Similarly, [Formula: see text] recovered by the application accurately predicted the actual [Formula: see text] for two-way experimental designs when summary statistics were available for both factors and the sub-plot factor in split-plot designs, irrespective of the post hoc mean separation test. The MSE FINDR Shiny application, documentation, and an accompanying tutorial are hosted at https://garnica.shinyapps.io/MSE_FindR/ and https://github.com/vcgarnica/MSE_FindR/ . With this tool, researchers can now easily estimate the within-study variance absent in published reports that nonetheless provide appropriate summary statistics, thus enabling the inclusion of such studies that would have otherwise been excluded in meta-analyses involving estimates of effect sizes based on a continuous response.