Logical contradictions in the One-way ANOVA and Tukey-Kramer multiple comparisons tests with more than two groups of observations

by   Vladimir Gurvich, et al.

We show that the One-way ANOVA and Tukey-Kramer (TK) tests agree on any sample with two groups. This result is based on a simple identity connecting the Fisher-Snedecor and studentized probabilistic distributions and is proven without any additional assumptions; in particular, the standard ANOVA assumptions (independence, normality, and homoscedasticity (INAH)) are not needed. In contrast, it is known that for a sample with k > 2 groups of observations, even under the INAH assumptions, with the same significance level α, the above two tests may give opposite results: (i) ANOVA rejects its null hypothesis H_0^A: μ_1 = … = μ_k, while the TK one, H_0^TK(i,j): μ_i = μ_j, is not rejected for any pair i, j ∈{1, …, k}; (ii) the TK test rejects H_0^TK(i,j) for a pair (i, j) (with i ≠ j) while ANOVA does not reject H_0^A. We construct two large infinite pseudo-random families of samples of both types satisfying INAH: in case (i) for any k ≥ 3 and in case (ii) for some larger k. Furthermore, in case (ii) ANOVA, being restricted to the pair of groups (i,j), may reject equality μ_i = μ_j with the same α. This is an obvious contradiction, since μ_1 = … = μ_k implies μ_i = μ_j for all i, j ∈{1, …, k}. Similar contradictory examples are constructed for the Multivariable Linear Regression (MLR). However, for these constructions it seems difficult to verify the Gauss-Markov assumptions, which are standardly required for MLR.


page 1

page 2

page 3

page 4


Optimal Group-Sequential Tests with Groups of Random Size

We consider sequential hypothesis testing based on observations which ar...

The Merging Path Plot: adaptive fusing of k-groups with likelihood-based model selection

There are many statistical tests that verify the null hypothesis: the va...

A Permutation Test on Complex Sample Data

Permutation tests are a distribution free way of performing hypothesis t...

Some multivariate goodness of fit tests based on data depth

Using the fact that some depth functions characterize certain family of ...

Asymptotic Invariance and Robustness of Randomization Tests

Proper testing of hypotheses requires adherence to the relevant assumpti...

K-sample omnibus non-proportional hazards tests based on right-censored data

This work presents novel and powerful tests for comparing non-proportion...

A reversal phenomenon in estimation based on multiple samples from the Poisson--Dirichlet distribution

Consider two forms of sampling from a population: (i) drawing s samples ...

Please sign up or login with your details

Forgot password? Click here to reset