A More Powerful Familywise Error Control Procedure for Evaluating Mean Equivalence

Abstract

When one wishes to show that there are no meaningful differences between two or more groups, equivalence tests should be used, as a nonsignificant test of mean difference does not provide evidence regarding the equivalence of groups. When conducting all possible post-hoc pairwise comparisons, C, Caffo, Lauzon and Rohmel (2013) suggested dividing the alpha level by a correction of k2/4, where k is the number of groups to be compared, however this procedure can be conservative in some situations. This research proposes two modified stepwise procedures, based on this correction of k2/4, for controlling the familywise Type I error rate. Using a Monte Carlo simulation method, we show that, across a variety of conditions, adopting a stepwise procedure increases power, particularity when a configuration of means has greater than C - k2/4 power comparisons, while maintaining the familywise error rate at or below . Implications for psychological research and directions for future study are discussed.