Analyzing Nested Experimental Designs – A User-Friendly Resampling Method to Determine Experimental Significance

Abstract

AbstractWe report Hierarch, a Python package to perform hypothesis tests and compute confidence intervals on hierarchical experimental designs. Using a combination of permutation resampling and bootstrap aggregation, Hierarch can be used to perform hypothesis tests that maintain nominal Type I error rates and generate confidence intervals that maintain the nominal coverage probability without making distributional assumptions about the dataset of interest. Hierarch makes use of the Numba JIT compiler to reduce p-value computation times to under one second for typical datasets in biomedical research. Hierarch also enables researchers to construct user-defined resampling plans that take advantage of Hierarch’s Numba-accelerated functions. Hierarch is freely available as a Python package at https://github.com/rishi-kulkarni/hierarch.