Are these two groups of scores significantly different ? A Bayesian approach
A common statistical inference problem in experimental psychology involves deciding whether or not there are differences in the means and variances of two groups of scores. Standard current practice for this problem is to use Null Hypothesis Significance Testing (NHST) methods, such as the t-test and Levene’s test. There are, however, serious theoretical and practical flaws in NHST, particularly in terms of its violation of the likelihood principle. Accordingly, this paper develops an alternative method using Bayesian statistical inference. The basic idea is to consider four different models, each of which assumes the groups scores comes from a Gaussian distribution, but which make different assumptions about the equality of means and variances across the groups. By finding maximum likelihood parameter values for these models, it is possible to calculate Bayesian Information Criterion values for each model, and hence estimate the Bayes Factors between each pair of models. The Bayes Factors quantify the evidence provided by the data in a naturally meaningful and interpretable way, allowing the assessment of whether differences are “significant” to be made in terms of the standards of scientific evidence required for the research problem at hand. An illustrative application of this Bayesian approach to real data is presented. The extension of the approach to cases where data have different distributions is discussed, as is the possibility of using Bayesian statistical inference for other standard problems of data analysis in psychology. ∗I thank Nick Burns, Daniel Navarro and Douglas Vickers for helpful comments, and Nick Burns for supplying the window tinting data. Correspondence should be addressed to: Michael D. Lee, Department of Psychology, University of Adelaide, SA 5005, AUSTRALIA. Telephone: +61 8 8303 6096, Facsimile: +61 8 8303 3770, E-mail: michael.lee@psychology.adelaide.edu.au, URL: http://www.psychology.adelaide.edu.au/members/staff/michaellee/homepage