A discussion and evaluation of statistical procedures used by JIMB authors when comparing means

Abstract

Abstract   Out of the 166 articles published in Journal of Industrial Microbiology and Biotechnology (JIMB) in 2019–2020 (not including special issues or review articles), 51 of them used a statistical test to compare two or more means. The most popular test was the (Standard) t-test, which often was used to compare several pairs of means. Other statistical procedures used included Fisher's least significant difference (LSD), Tukey's honest significant difference (HSD), and Welch's t-test; and to a lesser extent Bonferroni, Duncan's Multiple Range, Student–Newman–Keuls, and Kruskal–Wallis tests. This manuscript examines the performance of some of these tests with simulated experimental data, typical of those reported by JIMB authors. The results show that many of the most common procedures used by JIMB authors result in statistical conclusions that are prone to have large false positive (Type I) errors. These error-prone procedures included the multiple t-test, multiple Welch's t-test, and Fisher's LSD. These multiple comparisons procedures were compared with alternatives (Fisher–Hayter, Tukey's HSD, Bonferroni, and Dunnett's t-test) that were able to better control Type I errors. Non-technical summary The aim of this work was to review and recommend statistical procedures for Journal of Industrial Microbiology and Biotechnology authors who often compare the effect of several treatments on microorganisms and their functions.