Overview of Bayesian Statistics

Bayesian statistics is becoming a popular approach to handling complex statistical modeling. This special issue of Evaluation Review features several Bayesian contributions. In this overview, I present the basics of Bayesian inference. Bayesian statistics is based on the principle that parameters have a distribution of beliefs about them that behave exactly like probability distributions. We can use Bayes' Theorem to update our beliefs about values of the parameters as new information becomes available. Even better, we can make statements that frequentists do not, such as "the probability that an effect is larger than 0 is .93," and can interpret 95% (e.g.) intervals as people naturally want, that there is a 95% probability that the parameter is in that interval. I illustrate the basic concepts of Bayesian statistics through a simple example of predicting admissions to a PhD program.