Appropriate Statistical Analysis for Two Independent Groups of Likert-Type Data

The objective of this research was to determine the robustness and statistical power of three different methods for testing the hypothesis that ordinal samples of five and seven Likert categories come from equal populations. The three methods are the two sample t-test with equal variances, the Mann-Whitney test, and the Kolmogorov-Smirnov test. In additional, these methods were employed over a wide range of scenarios with respect to sample size, significance level, effect size, population distribution, and the number of categories of response scale. The data simulations and statistical analyses were performed by using R programming language version 2.13.2. To assess the robustness and power, samples were generated from known distributions and compared. According to returned p-values at different nominal significance levels, empirical error rates and power were computed from the rejection of null hypotheses.Results indicate that the two sample t-test and the Mann-Whitney test were robust for Likert-type data. Also the t-test performed the best to control of Type I error for both 5-point and 7-point Likert scale. For the Kolmogorov-Smirnov test, the Type I error rate was not controlled for all circumstances. This means the testing procedure computed from R was not robust for the ordinal Likert-type data because the Type I error rate of this test was lower than the minimum of the robustness criteria. Therefore, the Kolmogorov-Smirnov test was quite conservative. For the statistical power of the test, the Mann-Whitney test was the most powerful for most of the distributions, especially under highly skewed and bimodal distributions. The t-test obtained high statistical power or close to the power from the Man-Whitney test under the uniform, moderate skewed or symmetric distribution with large location shift.