Null Hypothesis Significance Testing: A Short Tutorial
Fleeting- Référence externe : https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5635437/
Null hypothesis significance testing: a short tutorial
NHST is a method of statistical inference by which an experimental factor is tested against a hypothesis of no effect or no relationship based on a given observation
only the null-hypothesis is tested, and therefore p-values are meant to be used in a graded manner to decide whether the evidence is worth additional investigation and/or replication
no isolated experiment, however significant in itself, can suffice for the experimental demonstration of any natural phenomenon’
The p-value is not an indication of the strength or magnitude of an effect.
the p-value is not informative on the effect itself
In low powered studies (typically small number of subjects), the p-value has a large variance across repeated samples, making it unreliable to estimate replication
The p-value is not the probability of the null hypothesis p(H0), of being true
This common misconception arises from a confusion between the probability of an observation given the null p(Obs≥t|H0) and the probability of the null given an observation p(H0|Obs≥t) that is then taken as an indication for p(H0)
The figure was prepared with G-power for a one-sided one-sample t-test, with a sample size of 32 subjects, an effect size of 0.45, and error rates alpha=0.049 and beta=0.80. In Fisher’s procedure, only the nil-hypothesis is posed, and the observed p-value is compared to an a priori level of significance. If the observed p-value is below this level (here p=0.05), one rejects H0. In Neyman-Pearson’s procedure, the null and alternative hypotheses are specified along with an a priori level of acceptance. If the observed statistical value is outside the critical region (here [-∞ +1.69]), one rejects H0.
- IIUC, Fisher’s p-values tests H0’s significance while Neaman&Pearson’s alĥa, beta compares the relative significance of H1
there is a profound difference between accepting the null hypothesis and simply failing to reject it
absence of evidence is not evidence of absence
To make a statement about the probability of a parameter of interest (e.g. the probability of the mean), Bayesian intervals must be used.
what is the goal of a scientific experiment at hand? If the goal is to establish a discrepancy with the null hypothesis and/or establish a pattern of order, because both requires ruling out equivalence, then NHST is a good tool
- Occam’s rasor
If the goal is to test the presence of an effect and/or establish some quantitative values related to an effect, then NHST is not the method of choice since testing is conditioned on H0
no isolated experiment, however significant in itself, can suffice for the experimental demonstration of any natural phenomenon
no single value (being p-values, Bayesian factor or else) can be used support or invalidate a theory
one cannot predict and/or discuss quantities without accounting for variability