The Effect of Non-normality on the Power of Randomization Tests: A Simulation Study Using Normal Mixtures

摘要

In this paper we evaluate the impact of non-normally on the power of randomization tests for two independent groups with the fifteen densities used in Marron and Wand [19] simulation study, which can all be written as normal mixtures and are believed to model many real data situations. We evaluate the power of the randomization test, and also the power of the Student-t test, as a comparison standard, with data simulated from the fifteen Marron-Wand distributions, for 81 values of effect size (from -4.0 to 4.0, by steps of 0.1) and balanced samples of 8, 16 and 32 elements. For each situation, using modules written in R [27], we have generated 20,000 samples and, for each of these, the power of the randomization tests was estimated using 1,000 data permutations. We set the value of Type I error probability at 0.05. In general, the results show that non-normality has a moderate influence on the power of the randomizations tests and that this influence reduces with increasing sample size. When we compare the non-normal distributions with the Gaussian, in terms of power, the differences range from approximately -0.05 to 0.05 (with an exception: in the case of one of the distributions, the differences range from 0.058 to 0.362). Concerning the comparison of the randomization test with the Student-t test, it was found that they have similar power, with some advantage to the former. In general, the differences in power are insignificant (from -0.013 to 0.015), but in one case, that of a non-normal distribution with outliers, the gains reach a maximum of 0.106. It is important to note that our results were obtained with balanced groups and with the same distribution for both groups. They can not therefore be generalized to situations where the groups are not balanced, have different distributions or are heteroscedastic.