Bayesian Statistics

Mike's Notes

A great introduction to Bayesian Statistics. With example workbooks. All free, including an Excel Resource Pack. The Real Statistics website by Charles Zaiontz is a fantastic resource.

Resources

• https://real-statistics.com/bayesian-statistics/
• https://real-statistics.com/bayesian-statistics/bayesian-statistics-for-binomial-distributed-data/analytic-approach-binomial-data/beta-conjugate-prior/
• https://real-statistics.com/free-download/
• https://en.wikipedia.org/wiki/Bayesian_inference
• https://en.wikipedia.org/wiki/Bayes%27_theorem

References

• Reference

Repository

• Home > Ajabbi Research > Library >
• Home > Handbook > 

Last Updated

20/12/2025

Bayesian Statistics

By: Charles Zaiontz

Real Statistics: 01/05/2021

Dr. Charles Zaiontz has a PhD in mathematics from Purdue University and has taught as an Assistant Professor at the University of South Florida as well as at Cattolica University (Milan and Piacenza) and St. Xavier College (Milan).

Bayesian statistics uses an approach whereby beliefs are updated based on data that has been collected. This can be an iterative process, whereby a prior belief is replaced by a posterior belief based on additional data, after which the posterior belief becomes a new prior belief to be refined based on even more data. The initial prior belief in this series may be based on intuition, previous studies, experience, etc.

In inferential statistics, we commonly test hypotheses, estimate parameters, and make predictions. In the traditional approach to statistics, commonly called the frequentist approach, parameters are constants whose values we aim to discern. Bayesian statistics uses a different approach: we treat these parameters as variables that have a probability distribution.

Topics

• Introduction
  • Example
  • Hypothesis Testing
• Binomial Data
  • Grid Approximation
  • Analytic Approach
  • Markov-Chain Monte Carlo
  • Two-Sample Case
• Normal Data
  • Conjugate Priors
  • Monte Carlo Simulation
  • Gibbs Sampler
  • Hypothesis Testing
• Conjugate Pairs
• Non-informative Priors
• Correlation Testing
• Independence Testing
• Non-parametric Testing
• Monte Carlo Integration

References

1. Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., Rubin, D. B. (2014) Bayesian data analysis, 3rd Ed. CRC Press
   https://statisticalsupportandresearch.files.wordpress.com/2017/11/bayesian_data_analysis.pdf
2. Marin, J-M and Robert, C. R.  (2014) Bayesian essentials with R. 2nd Ed. Springer
   https://www.springer.com/gp/book/9781461486862
3. Jordan, M. (2010) Bayesian modeling and inference. Course notes
   https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/index.html
4. Lee, P. M. (2012) Bayesian statistics an introduction. 4th Ed. Wiley
   https://www.wiley.com/en-us/Bayesian+Statistics%3A+An+Introduction%2C+4th+Edition-p-9781118332573