1. Probability and Bayes’ Theorem
(1) Probability
a) Classical and Frequentist Probability
Statistics : ‘study of uncertainty’
There are 3 different frameworks, under which we define “probability”
- Classical Framework
- Frequentist Framework
- Bayesian Framework
[ Classical Framework ]
outcomes that are equally likely have equal probabilities
( ex. fair die with number 1~6 -> so the prob of each outcome is one sixth )
[ Frequentist Framework ]
-
find P(data model) (= assuming the data with the model that we have) - model : certian / data : uncertain
- “objective” in how it defines probability
- “relative” frequency
- outcome : point estimates with standard error
- parameter of the model : fixed & unkown constant
[ Bayesian Framework ]
-
find P(model data) (= assuming the model with the data that we have) - model : uncertain / data : ground truth
- “subjective” in how it defines probability
- degree of belief
- outcome : posterior distribution
- parameter of the model : random variables
https://agostontorok.github.io/public/img/b_vs_f/pic_1.JPG
These are the brief summary of three frameworks with probability, and I’m going to focus on the “Bayesian Framework”.
b. Bayesian Probability and Coherence
Bayesian perspective is ‘personal’ perspective. A subjective approach to probability can lead to a more intuitive result in many cases, compared to Frequentists.
Let’s take an example.
You made a bet that you win 10$ if it rains tomorrow, and lose 1$ if it doesn’t.
What is the a probability(=p) that it will rain tomorrow, from your own belief?
(10 * p) + ((-1)*(1-p)) = 0
Therefore, p is 1/9. And this result will depend on who the better will be.
(of course, as long as you are not willing to lose money in this bet)
But don’t forget about coherence! Probability should follow all the rules of probability, or else you’ll
lose money on that kind of bet!
(2) Bayes’ Theorem
I will not cover much about Bayes’ Theorem, assuming that everyone has already learned about it.
The formula looks like the below.
https://pbs.twimg.com/media/DE_ZYUOUQAAltjc.jpg
To summarize into one sentence, we can use ‘Bayes’ Theorem’ when we want to calculate the posterior probability, when we have the likelihood and the other prior probabilities.