Validity of Randomization Tests for one-Subject Experiments

Abstract

Valid Statistical tests for one-subject experiments are necessary to justify Statistical inferences and to ensure the acceptability of research reports to a wide range of journals and readers. The validity of randomization tests for one-subject experiments is examined in this paper. A randomization test is a procedure for determining significance in the following manner. A test statistic (e.g., t or F) is computed for a set of research data. The value of the test statistic is called the “obtained test statistic value.” The data are then divided repeatedly, and the test statistic is computed for each data division. If the proportion of the data divisions giving a test statistic value as large as the obtained test statistic value is no greater than α, the test statistic is significant at the α level. Any Statistical test is said to be a randomization test when the significance of its test statistic is determined by the randomization test procedure. Determination of significance by the randomization test procedure permits the valid application of any Statistical test, whether it be as simple as a t test or as complex as factorial multivariate analysis of variance, for one-subject as well as multiple-subject experiments. For the randomization test procedure to be valid for a one-subject experiment, there must be random assignment of treatment times to treatments (i.e., random determination of when each treatment is to be given); specification, before observing the data, of the test statistic and the criterion to be employed for discarding data; and, in the determination of significance, division of the data in a manner consistent with the random assignment procedure.