Professor Neil Pearce, Department of Medical Statistics, London School of Hygiene and Tropical Medicine
Title: Current controversies in causal inference
Marie-Abèle Bind, Department of Statistics, Harvard University
Title: 21st century causal inference that capitalizes on classical design of experiments and modern computing.
Abstract: In randomized experiments, Fisher-exact p-values are available and should be used to help evaluate results rather than the more commonly reported asymptotic p-values. One reason is that using the latter can effectively alter the question being addressed by including irrelevant distributional assumptions. The Fisherian statistical framework, proposed in 1925, calculates a p-value in a randomized experiment by using the actual randomization procedure that led to the observed data. Here, we illustrate this Fisherian framework in a crossover randomized experiment. We focus on 10 outcomes that illustrate important differences between the asymptotic and Fisher tests for the null hypothesis of no ozone effect. For some outcomes, the traditional p-value based on the approximating asymptotic Student’s t distribution substantially subceeded the minimum attainable Fisher-exact p-value. For the other outcomes, the Fisher-exact null randomization distribution substantially differed from the bell-shaped one assumed by the asymptotic t test. Our conclusions: When researchers choose to report p-values in randomized experiments, 1) Fisher-exact p-values should be used, especially in studies with small sample sizes, and 2) the shape of the actual null randomization distribution should be examined for the recondite scientific insights it may reveal.