Permutation testing for thick data when the number of variables is much greater than the sample size: recent developments and some recommendations

Abstract

AbstractIn many scientific disciplines datasets contain many more variables than observational units (so-called thick data). A common hypothesis of interest in this setting is the global null hypothesis of no difference in multivariate distribution between different experimental or observational groups. Several permutation-based nonparametric tests have been proposed for this hypothesis. In this paper we investigate the potential differences in performance between different methods used to test thick data. In particular we focus on an extension of the Nonparametric combination procedure (NPC) proposed by Pesarin and Salmaso, a rank-based approach by Ellis, Burchett, Harrar and Bathke, and a distance-based approach by Mielke. The effect of different combining procedures on the NPC is also explored. Finally, we illustrate the use of these methods on a real-life dataset.