The Vector Analytics Project
Advancing the use of effect size statistics
The idea that statistical significance should be abandoned has received a lot of attention recently but it is not new, and efforts to replace p-value has been underway for some time. Perhaps the largest effort involves the development of effect size statistics. However, the large number of alternative effect size measures that have been proposed (Huberty, 2002) indicates that there are unresolved problems in this field as well. Only a small part of the large literature on these problems is cited here.
The statistics community issues new warnings about the misleading properties of p-value
New and old efforts to develop effect size statistics
References
- Agresti, Alan. (2001). "Exact inference for categorical data: recent advances and continuing controversies". Statistics in Medicine. 20 (17-18): 2709–2722. doi:10.1002/sim.738
- Cumming, Geoff. (2012). Understanding The New Statistics. New York, NY: Routledge. doi:10.4324/9780203807002
- Gelman, Andrew; Loken, Eric. (2014). "The Statistical Crisis in Science". American Scientist. 102 (6): 460. DOI: 10.1511/2014.111.460
- Goodman, Leo A.; Kruskal, William H. (1954). "Measures of Association for Cross Classifications". Journal of the American Statistical Association. 49 (268): 732–764. doi:10.2307/2281536
- Grissom, Robert J.; Kim, John J. (2012). Effect Sizes for Research: Univariate and Multivariate Applications. (2nd ed.). New York, NY: Routledge/Taylor & Francis Group. doi:10.4324/9780203803233
- Hand, David J. (2006). "Classifier Technology and the Illusion of Progress." Statistical Science. 21 (1): 1–14. doi:10.1214/088342306000000060
- Hedrick, Philip W. (1987). "Gametic disequilibrium measures: proceed with caution". Genetics. 341 (2):, 331-341.
- Huberty, Carl J. (2002). "A History of Effect Size Indices". Educational and Psychological Measurement. 62 (2): 227-240. doi:10.1177/0013164402062002002
- Kelley, Ken, and Kristopher J. Preacher. (2012). “On Effect Size.” Psychological Methods 17 (2): 137–52. doi:10.1037/a0028086
- Leek, Jeff.; McShane, Blakeley B.; Gelman, Andrew; Colquhoun, David; Nuijten, Michèle B.; Goodman, Steven N. (2017). "Five ways to fix statistics". Nature. 551 (7682): 557-559. doi:10.1038/d41586-017-07522-z
- McCloskey, Deirdre; Ziliak, Steve. (2008). The Cult of Statistical Significance. Ann Arbor, MI: University of Michigan Press. doi:10.3998/mpub.186351
- Olivier, Jake; Bell, Melanie L. (2013). "Effect sizes for 2x2 contingency tables". PLoS ONE. 8 (3): e58777. doi:10.1371/journal.pone.0058777
- Nakagawa, Shinichi; Cuthill, Innes C. (2007). "Effect size, confidence interval and statistical significance: a practical guide for biologists". Biological Reviews of the Cambridge Philosophical Society. 82 (4): 591–605. doi:10.1111/j.1469-185X.2007.00027.x
- Stark, Philip B.; Saltelli, Andrea. (2018). “Cargo-Cult Statistics and Scientific Crisis”. Significance 15 (4): 40–43. doi:10.1111/j.1740-9713.2018.01174.x
- Warrens, Matthijs J. (2008). "On Association Coefficients for 2 x 2 Tables and Properties That Do Not Depend on the Marginal Distributions". Psychometrika. 73 (4): 777-789. doi:10.1007/s11336-008-9070-3
- Wasserstein, Ronald L.; Lazar, Nicole A. (2016) "The ASA's Statement on p-Values: Context, Process, and Purpose". The American Statistician. 70(2): 129-133. doi:10.1080/00031305.2016.1154108