Advancing the use of effect size statistics

The Vector Analytics Project

Developing algorithms and making effect size statistics accessible.
Inverse problems in covariate data analysis: In this paper, we demonstrate that there are distinct mathematical problems associated with numeric (R x R), categorical (C x C), and mixed categorical-numeric (C x R) data.
A parametric framework for multidimensional linear measurement error regression: In this paper, we discuss the fact that statistical measures of linear dependence, including covariance, correlation, and regression slope, are subject to the chain rule.
Nonoverlap proportion and the point-biserial association problem: Expository article published by ResearchOutreach.org.
Nonoverlap proportion and the representation of point-biserial variation: In this paper, we discuss the fact that statistics methodology is subject to the general physical principle that it is necessary to account for all of the degrees of freedom when studying a quantitative phenomenon. We develop a computational framework for generating the statistical parameters for point-biserial variation.
Factoring a 2 x 2 contingency table: In this paper, we show that a contingency table can be expressed as a product of marginal sum and proportion matrices. We also identify effect size measures for a 2 x 2 table that are invariant to variation in the marginal sums. The latter property is important for obtaining reproducible results in the analysis of categorical data.
The National Academy of Sciences 2017 Colloqium on Reproducibility of Research: Issues and Proposed Remedies

Advancing the use of effect size statistics

The idea that statistical significance should be abandoned has received a lot of attention recently but it is not new, and efforts to replace p-value has been underway for some time. Perhaps the largest effort involves the development of effect size statistics. However, the large number of alternative effect size measures that have been proposed (Huberty, 2002) indicates that there are unresolved problems in this field as well. Only a small part of the large literature on these problems is cited here.

The statistics community issues new warnings about the misleading properties of p-value

Five Ways to Fix Statistics

The Statistical Crisis in Science

Moving to a World Beyond "p < 0.05"

New and old efforts to develop effect size statistics

On Effect Size

Introduction to the New Statistics

Stephen T. Ziliak's Library of Oomph & Precision

A Gentle Introduction to Effect Size Measures in Python

Jacob Cohen (statistician)

References

Agresti, Alan. (2001). "Exact inference for categorical data: recent advances and continuing controversies". Statistics in Medicine. 20 (17-18): 2709–2722. doi:10.1002/sim.738
Cumming, Geoff. (2012). Understanding The New Statistics. New York, NY: Routledge. doi:10.4324/9780203807002
Gelman, Andrew; Loken, Eric. (2014). "The Statistical Crisis in Science". American Scientist. 102 (6): 460. DOI: 10.1511/2014.111.460
Goodman, Leo A.; Kruskal, William H. (1954). "Measures of Association for Cross Classifications". Journal of the American Statistical Association. 49 (268): 732–764. doi:10.2307/2281536
Grissom, Robert J.; Kim, John J. (2012). Effect Sizes for Research: Univariate and Multivariate Applications. (2nd ed.). New York, NY: Routledge/Taylor & Francis Group. doi:10.4324/9780203803233
Hand, David J. (2006). "Classifier Technology and the Illusion of Progress." Statistical Science. 21 (1): 1–14. doi:10.1214/088342306000000060
Hedrick, Philip W. (1987). "Gametic disequilibrium measures: proceed with caution". Genetics. 341 (2):, 331-341.
Huberty, Carl J. (2002). "A History of Effect Size Indices". Educational and Psychological Measurement. 62 (2): 227-240. doi:10.1177/0013164402062002002
Kelley, Ken, and Kristopher J. Preacher. (2012). “On Effect Size.” Psychological Methods 17 (2): 137–52. doi:10.1037/a0028086
Leek, Jeff.; McShane, Blakeley B.; Gelman, Andrew; Colquhoun, David; Nuijten, Michèle B.; Goodman, Steven N. (2017). "Five ways to fix statistics". Nature. 551 (7682): 557-559. doi:10.1038/d41586-017-07522-z
McCloskey, Deirdre; Ziliak, Steve. (2008). The Cult of Statistical Significance. Ann Arbor, MI: University of Michigan Press. doi:10.3998/mpub.186351
Olivier, Jake; Bell, Melanie L. (2013). "Effect sizes for 2x2 contingency tables". PLoS ONE. 8 (3): e58777. doi:10.1371/journal.pone.0058777
Nakagawa, Shinichi; Cuthill, Innes C. (2007). "Effect size, confidence interval and statistical significance: a practical guide for biologists". Biological Reviews of the Cambridge Philosophical Society. 82 (4): 591–605. doi:10.1111/j.1469-185X.2007.00027.x
Stark, Philip B.; Saltelli, Andrea. (2018). “Cargo-Cult Statistics and Scientific Crisis”. Significance 15 (4): 40–43. doi:10.1111/j.1740-9713.2018.01174.x
Warrens, Matthijs J. (2008). "On Association Coefficients for 2 x 2 Tables and Properties That Do Not Depend on the Marginal Distributions". Psychometrika. 73 (4): 777-789. doi:10.1007/s11336-008-9070-3
Wasserstein, Ronald L.; Lazar, Nicole A. (2016) "The ASA's Statement on p-Values: Context, Process, and Purpose". The American Statistician. 70(2): 129-133. doi:10.1080/00031305.2016.1154108