"... if statisticians cannot agree how to analyse even a 2×2 table, with no one approach being obviously ‘best’, what hope is there for a consensus on more complex analyses? The probability is probably decreasing to 0 with time, ..."Agresti, 2001

The Vector Analytics Project

Advancing the use of effect size statistics

The idea that statistical significance should be abandoned has received a lot of attention recently but it is not new, and efforts to replace p-value has been underway for some time. Perhaps the largest effort involves the development of effect size statistics. However, the large number of alternative effect size measures that have been proposed (Huberty, 2002) indicates that there are unresolved problems in this field as well. Only a small part of the large literature on these problems is cited here.

The statistics community issues new warnings about the misleading properties of p-value

New and old efforts to develop effect size statistics


  1. Agresti, Alan. (2001). "Exact inference for categorical data: recent advances and continuing controversies". Statistics in Medicine. 20 (17-18): 2709–2722. doi:10.1002/sim.738
  2. Cumming, Geoff. (2012). Understanding The New Statistics. New York, NY: Routledge. doi:10.4324/9780203807002
  3. Gelman, Andrew; Loken, Eric. (2014). "The Statistical Crisis in Science". American Scientist. 102 (6): 460. DOI: 10.1511/2014.111.460
  4. Goodman, Leo A.; Kruskal, William H. (1954). "Measures of Association for Cross Classifications". Journal of the American Statistical Association. 49 (268): 732–764. doi:10.2307/2281536
  5. Grissom, Robert J.; Kim, John J. (2012). Effect Sizes for Research: Univariate and Multivariate Applications. (2nd ed.). New York, NY: Routledge/Taylor & Francis Group. doi:10.4324/9780203803233
  6. Hand, David J. (2006). "Classifier Technology and the Illusion of Progress." Statistical Science. 21 (1): 1–14. doi:10.1214/088342306000000060
  7. Hedrick, Philip W. (1987). "Gametic disequilibrium measures: proceed with caution". Genetics. 341 (2):, 331-341.
  8. Huberty, Carl J. (2002). "A History of Effect Size Indices". Educational and Psychological Measurement. 62 (2): 227-240. doi:10.1177/0013164402062002002
  9. Kelley, Ken, and Kristopher J. Preacher. (2012). “On Effect Size.” Psychological Methods 17 (2): 137–52. doi:10.1037/a0028086
  10. Leek, Jeff.; McShane, Blakeley B.; Gelman, Andrew; Colquhoun, David; Nuijten, Michèle B.; Goodman, Steven N. (2017). "Five ways to fix statistics". Nature. 551 (7682): 557-559. doi:10.1038/d41586-017-07522-z
  11. McCloskey, Deirdre; Ziliak, Steve. (2008). The Cult of Statistical Significance. Ann Arbor, MI: University of Michigan Press. doi:10.3998/mpub.186351
  12. Olivier, Jake; Bell, Melanie L. (2013). "Effect sizes for 2x2 contingency tables". PLoS ONE. 8 (3): e58777. doi:10.1371/journal.pone.0058777
  13. Nakagawa, Shinichi; Cuthill, Innes C. (2007). "Effect size, confidence interval and statistical significance: a practical guide for biologists". Biological Reviews of the Cambridge Philosophical Society. 82 (4): 591–605. doi:10.1111/j.1469-185X.2007.00027.x
  14. Stark, Philip B.; Saltelli, Andrea. (2018). “Cargo-Cult Statistics and Scientific Crisis”. Significance 15 (4): 40–43. doi:10.1111/j.1740-9713.2018.01174.x
  15. Warrens, Matthijs J. (2008). "On Association Coefficients for 2 x 2 Tables and Properties That Do Not Depend on the Marginal Distributions". Psychometrika. 73 (4): 777-789. doi:10.1007/s11336-008-9070-3
  16. Wasserstein, Ronald L.; Lazar, Nicole A. (2016) "The ASA's Statement on p-Values: Context, Process, and Purpose". The American Statistician. 70(2): 129-133. doi:10.1080/00031305.2016.1154108