The effects of nonnormality on parametric, nonparametric and model comparison approaches to pairwise comparisons

Researchers in the behavioral sciences are often interested in comparing the means of several treatment conditions on a specific dependent measure. When scores on the dependent measure are not normally distributed, researchers must make important decisions regarding the multiple comparison strategy that is implemented. Although researchers commonly rely on the potential robustness of traditional parametric test statistics (e.g., t and F), these test statistics may not be robust under all nonnormal data conditions. This article compared strategies for performing multiple comparisons with nonnormal data under various data conditions, including simultaneous violations of the assumptions of normality and variance homogeneity. The results confirmed that when variances are unequal, use of the traditional two-sample t test can result in severely biased Type I and/or Type II error rates. However, the use of Welch’s two-sample test statistic with the REGWQ procedure, with either the usual means and variances or with trimmed means and Winsorized variances, resulted in good control of Type I error rates. The Kruskal-Wallis nonparametric statistic provided good Type I error control and power when variances were equal, although Type I error rates became severely inflated when variances were unequal. Furthermore, for researchers interested in eliminating intransitive decisions or comparing potential mean configuration models, a protected model-testing procedure suggested by Dayton provided good overall results.