T-Tests, Non-Parametric Tests, and Large Studies—a Paradox of Statistical Practice?

Bibliographic Data

ID23374386
AuthorsMorten Wang Fagerland (0000-0003-4535-5043, Oslo University Hospital, corresponding author)
Year2012
Volume12
Issue1
Pages78-78
Publication date2012-12-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueBMC Medical Research Methodology (JOURNAL)
Journal identifiersISSN: 1471-2288 • E-ISSN: 1471-2288
PublisherSpringer Science and Business Media LLC (PUBLISHER)
DOI10.1186/1471-2288-12-78
PMID22697476
PMCIDPMC3445820
OpenAlexW2144683894
LanguageEN
Citations received63
References cited14

BACKGROUND: During the last 30 years, the median sample size of research studies published in high-impact medical journals has increased manyfold, while the use of non-parametric tests has increased at the expense of t-tests. This paper explores this paradoxical practice and illustrates its consequences. METHODS: A simulation study is used to compare the rejection rates of the Wilcoxon-Mann-Whitney (WMW) test and the two-sample t-test for increasing sample size. Samples are drawn from skewed distributions with equal means and medians but with a small difference in spread. A hypothetical case study is used for illustration and motivation. RESULTS: The WMW test produces, on average, smaller p-values than the t-test. This discrepancy increases with increasing sample size, skewness, and difference in spread. For heavily skewed data, the proportion of p<0.05 with the WMW test can be greater than 90% if the standard deviations differ by 10% and the number of observations is 1000 in each group. The high rejection rates of the WMW test should be interpreted as the power to detect that the probability that a random sample from one of the distributions is less than a random sample from the other distribution is greater than 50%. CONCLUSIONS: Non-parametric tests are most useful for small studies. Using non-parametric tests in large studies may provide answers to the wrong question, thus confusing readers. For studies with a large sample size, t-tests and their corresponding confidence intervals can and should be used even for heavily skewed data.

Confidence interval · Econometrics · Mann–Whitney U test · Nonparametric statistics · Parametric statistics · Percentile · Sample (material) · Sample size determination · Skewness · Standard deviation · Statistical hypothesis testing · Statistical power · Statistical significance · Statistics · Test (biology) · Wilcoxon signed-rank test · Mathematics · Meta-analysis and systematic reviews · Statistical Methods and Bayesian Inference · Statistical Methods in Clinical Trials