Statistical Significance - Problems and Solutions?
There has been much recent discussion about problems associated with the use of 'statistical significance' and the resulting dichotomy; p<0.05 supports the presence of an effect, while p>0.05 supports no effect and the often very rigid interpretation. The p-value is a measure of 'strength of evidence' based on the data. Just because p>0.05 does not mean that there is 'no effect'; it simply means that the evidence to hand does not support an effect. A small sample size almost by definition, because of low power, will invariably mean that the result is 'non-significant', irrespective of whether in truth there is an effect of the targeted size.
Here are two articles, one from the American Statistical Association and the other from Nature that explain the issues and offer some quite radical solutions.
https://www.tandfonline.com/doi/full/10.1080/00031305.2016.1154108
https://www.nature.com/articles/d41586-019-00857-9
Here are two articles, one from the American Statistical Association and the other from Nature that explain the issues and offer some quite radical solutions.
https://www.tandfonline.com/doi/full/10.1080/00031305.2016.1154108
https://www.nature.com/articles/d41586-019-00857-9