Complaints about fishing for small P-values may be traced to the mid-19th century, and 120 years have passed since Pearson (1906, who coined “value of P”) warned against null misinterpretation of large P-values. Confidence intervals and Bayesian methods have been touted as cures for such problems, but have done little to mitigate the core problems behind statistics misinterpretation and abuse. That is unsurprising, given the problems arise from psychosocial forces that are assumed absent by statistical philosophies and methods, yet arise naturally in research and its reporting. Those forces include perverse incentives toward overstating certainty, producing desired inferences, and protecting prior commitments, all of which overlap and interact to distort reporting in ways beyond mere selection effects. These forces transform methods for “statistical inference” from tools of rational discourse into tools for rhetorical persuasion and deception (including self-deception). Informative prior distributions aggravate this problem. It is thus argued that statistical methodology needs to be reconstructed in ways that acknowledge these problems as well as other bias sources. The reconstruction advocated here restores detailed description of data generation as an essential foundation for all statistical analysis. It also inserts causal modeling and cognitive psychology as essential tools for evaluating “data quality” and for moving from description to explanation, inference, and decisions.
Sander Greenland is Emeritus Professor of Epidemiology and Statistics at the University of California, Los Angeles, and a Fellow of the American Statistical Association and the Royal Statistical Society. He has published over 500 articles and book chapters in epidemiology, statistics, and medicine, and given several hundred invited lectures, seminars and courses worldwide in epidemiologic and statistical methodology, with a focus on delineating and preventing misuse of statistical methods.
Member - FREE
Non-member - £10.00